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ABSTRACT 



This course workbook is designed to help students 
understand the principles of active learning, student assessment, and 
team training used in Arizuna State University*s core engineering 
curriculum. Eight sections focus on: (1) active learning, teaching 
and learning styles, and cooperative learning; (2) student 
assessment, levels of learning, and degrees of internalization, (3) a 
guide to self-evaluation and the documentation of educational states; 
(4) the use of teams; (5) team dynamics and effectiveness; (6) 
communication skills and teamwork; (7) tools for effective team 
decision-making; and (8) competency matrices for student 
self -evaluati on. (MDM) 
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Preface 



Most of you will find that this course is managed and delivered in a way that 
differs from your prior classroom experiences. Consequently, many of you may 
find the approach unusual and perhaps a few of you will find the approach 
disturbing. It is important that you be aware of (and eventually understand and 
perhaps value) the reasons for these differences in course management and 
delivery. The two most important considerations, the ones which most strongly 
motivate how the class is managed, are: 1) active student involvement and, 2) 
the role of (course) expectations. 
Active Student Involvement 

Haven't you always been involved, gone to class, done the assigned reading 
and work, and taken the tests? Of course you have, and you will continue to do 
so in this class; however, active student involvement goes beyond these 
activities. What involvement did you have in: 

• selecting the assignments or the topics to be covered; 

• improving the course in real time (i.e., while you were taking it); 

• deciding how the testing was to be done; 

• establishing course goals; 

• helping class colleagues to master the material? 

You have probably had little if any involvement 1n these activities which are the 
very activities that are included in the term active student involvement. 

Would you like to be actively involved? Probably yes, although you may wonder 
if this type of student involvement is educationally sound; will it degrade the 
class and the educational experience? The answer is an overwhelming NO. In 
fact, research has clearly shown that active involvement actually leads to a 
significant upgrade of most classes. Every current journal of education or 
educational research has at least one article extolling the benefits of some 
aspect of active student involvement. The bottom line is: the more you are 
involved in deciding how a course is managed (i.e., constructing your learning 
environment) the more you learn and the more you value the learning process. 

This body of evidence, collected and analyzed by educational experts cannot be 
ignored in engineering education any more than we can ignore the laws of 
physics; therefore, for the past two years, we have been actively changing our 
course management to incorporate these research findings. This has not 
been easy, it is one thing to agree philosophically with this data but an entirely 
different matter to actually incorporate it into my value structure, especially since 
we have spent virtually our entire life in a learning culture which says that we, 
as the professors, are right and make all the decisions. 

Considerable effort and hard work have gone into making the transformation in 
my classrooms; however, we have found that our efforts have not been one 
hundred percent successful. Some methods and activities that we have tried 
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have worked and the results have been fantastic; other methods and activities 
have not worked as well and when this happened the students tended to 
become frustrated (angry) and reluctant to continue trying to make the class 
work^. We have eliminated or modified those activities which have not worked; 
however, we have also added some new activities to provide additional 
opportunities for you to be significantly and actively involved in the class. 

In this class active involvement means you will: 

1. help select some of the problems to be studied during 
the semester 

2. make real time suggestions on how to improve the 
course 

3. help develop some of the course expectations 

4. help colleagues learn 

The Role of (Course) Expectations 

Expectations2 , We have them, you have them. Society has them; but what is 
their role in a course? Without expectations assessment is not possible . 
Expectations establish the baseline to which work is compared 
(benchmarked). When assessing work, the assessor's task is to determine if 
the work meets, exceeds, or falls short of expectations. Thus, assessment 
requires explicit expectations; however, you cannot do any work until you 
know the expectations. You have to know what you are trying to accomplish. 
You can no more prepare v/ork before knowing the expectations than you can 
prepare a design before knowing the requirements. 

Historically, classwork expectations were established after the work was 
completed (when it was no longer of use to you), established solely by the 
course instructor, and were often poorly explained or presented to you. Even 
when expectations were discussed, the discussion often took place at a level 
of detail that made the discussion virtually meaningless (e.g., / expect you to 
convince me; or, do a complete job). There is too much room for honest 
interpretation about what do a complete job might mean. Expectations such as 
do a complete job are a starting point but are no more useful than a design 
requirement of must be reliable. To be useful, expectations (and design 
requirements) must be defined at a level of detail that is either quantifiable or 
has little room for personal interpretation. 

We want to change this historical method of establishing expectations. First, 
we want you to be involved in establishing expectations. The work in 



^Thls is not a desirable situation and is the motivation for preparing these notes on the 
course. I want to lessen these feelings of frustration with new things that do not seem to 
work so that we can work out solutions and improvements and not just fee! bad. Active 
involvement requires suggestions from you on how to improve the course. 

2We are using the term expectations to refer to things which will occur (be present) and not to 
refer to things which we hope will occur (be present). 
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educational research shows that students produce much better work when 
they are involved in establishing the expectations for the work. Second, we 
want the expectations to be defined at an operational level, one you can use 
when developing and assessing your work. And third, we want these 
expectations to be established before the work is started (or at least well before 
it is completed). This gives you the target and helps keep the target from 
moving. 

This workbook contains a set of slides for the first few class periods in which 
teams, team building and assessment are discussed. Section C contains a 
guide to help you do assessment. We are not expecting you to become 
overnight experts at any of this material but we are hoping that by the end of the 
semester you will be willing lo participate in an active student classroom. 

So kick back, get ready to work and let's have a fun semester. 

LB & BWMcN 
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Schedule 



for Engineering Core Active Learning and Assessmenfl 



Total Time 


Time 


Class Period 1 : Active Learning 


10 


10 


Contact Before Work 


20 


10 


Getting Started 


40 


20 


Introduction to Active Learning 


60 


20 


Pigs Face Jigsaw 


70 


10 


Break 



Total Time 


Time 


Class Period 2 : Assessment 


5 


5 


Contact Before Work 


13 


8 


Getting Started 


28 


15 


introduction to Assessment 


28 




Levels of Learning Jigsaw 


62 


34 


Becoming an Expert 


72 


10 


Break 


90 


18 


Delivering Tutorials 


110 


20 


Introduction to Competency Matrices 
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Schedule for Engineering Core Team Training 



Total Time 


Time 


Class Period 3 : Jigsaw Part 1 


10 


10 


Clearing and Getting Started 


20 


10 


Teams Lecturette 


40 


20 


Learning the Material 


75 


35 


Meetings, Bloody Meetings Video 


85 


10 


Break 


105 


20 


Develop Tutorial 


110 


5 


Session Process Check (+/delta) 






Total Time 


Time 


Class Period 4 : Jigsaw Part 2 


5 


5 


Clearing and Forming New Teams 


40 


35 


Teach the Material ( 5 Topics x 7 Minutesn"opic ) 


55 


15 


Jigsaw Process Check (Likert Scale) 


65 


10 


Break 




5 


Academic Journal Introduction 


Total Time 


Time 


Class Period 4 : Team Norms Part 1 


85 


15 


Communication Skills & Norms Lecturette 


110 


25 


Team Player Video 






1 Total Time 


Time 


Class Period 5 : Team Norms Part 2 


10 


10 


Clearing and Review 


35 


25 


Communications Promoters & Barriers 


70 


35 


Developing Norms 


80 


10 


Break 


95 


15 


Discussion and Constructive Feedback Practice 


100 


5 


Session Process Check (+/delta) 


110 


10 


Academic Journal Entry 



Class Schedule for Workbook Training ^ Spring '95 : 1 /4/95 
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m %<- Learning Environment 

- Active (as opposed to Passive) 



today's Class Activities 



Before Class Starts 

- find a table (area) with no more than five people 

- try to find a table at which you know at most one person 

❖ Contact Before Work (10 minutes) 

- introduce yourself 

- find out who traveled the farthest to class today 

❖ Getting Started (10 minutes) 

❖ Active Learning (40 minutes) 

- short introductory lecture (20 minutes) 

- jigsaw (PIGS Face) (20 minutes) 

❖ Break (10 minutes) 



- Group or team based 

- Workshop facilitators (not Lecturers) 

❖ Purpose and Expected Participant Outcomes 

- Learrt Major Prirtciples of Active Learning 

- Practice Some Acetive Learning 
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{JJ^^o^ on Facilitator^ Signal 





The facilitator needs your attention: 

• Raise your hands to inform your neighbors 

• Finish your sentence 

• Do NOT finish your paragraph 

• Turn toward the Facilitator 




Someone will be assigned to be the Issue Bin Collector 
The following iss^^es will be assigned to the Issue Bin: 

- topics that will or may be addressed later 

- questions that can or should be deferred until the end of the 
workshop 

- items that can or should be the subject of future workshops 

Paraphrase the issue and record on the board or a piece of paper 
which is always visible 

At the conclusion of the session or workshop^ the issues in the 
issue bin are brought out, one at a time, and discussed to see if 
they are still issues. 
♦ Any issues which remain after the discussion must be addressed 
in a future workshop. 
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Code Of Cooperation 



1. EVERY member is responsible for the team's i>rogress and success 

2. Attend ail team meetings and be on time 

3. Come prepared. 

4. Cany out assignments on sdicdule. 

5. Listen to and show respect for the contributions of other membets; be an active listener. 

6. CONSTRUCTIVELY criticize ideas, not persons. 

7. Resolve conflicts constructively. 

8. Pay attention, avoid disruptive behavior. 

9. Avoid disruptive side conversations. 

10. Only one person speaks at a time. 

11. Everyone participates, no one dominates. 

12. Be sucdnct, avoid long anecdotes and examples. 

13. No rank in the room. 

14. Respect those not present 

15. Ask questions when you do not understand. 

16. Attend to your personal comfort needs at any time but minimize team disruption. 

17. HAVE FUN Mi 

18. ? 



adapted trom the Boeing Airpjane Group team Member Training Manual 




A Proposed Learning Culture 



MEETINGS IN CLASS ( F - F ) 
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MEETINGS OUTSIDE OF CLASS ( F - F ) 





Active (Cooperative) Learning 



❖ A technique used in the classroom which 
employs student-student and student-facilitator 
(faculty) interaction in various forms to convert 
the learning environment from PASSIVE to 
ACTIVE 



❖ Enhances learning 

❖ Substantially improves retention 

❖ Increases in value as the material increases in 
conceptual difficulty 




Urns and Team Training 



❖ Is used to enhance the performance of a *group' 

(i.e., group ==> Team) 

❖ Applies both INSIDE and OUTSIDE the classroom 

❖ Applies to both faculty and students 



Does NOT just happen; 
training is required I 
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Quality Principles 



❖ standardized processes, elementary tools, and a 
philosophy employed to induce a systemic change 
in the learning environment 

❖ Elements include: 

- continuous improvement of the process of education 
guided by timely feedback from customers 

- criterion-based assessment (e.g., a competency matrix) 

- testing as a feedback mechanism, 

not as a method for introducing variance 

- trust rather than fear 

- cooperation, not competition, at all levels 

- developing intrinsic, not extrinsic motivation 

- an integrated curriculum 

- patience and persistence 



. We (w[d these tnitfts aCso to Be seCf evident: 



❖ Life is far (earning. 

❖ (CHfferent peopU team differcntty and fiave 
a rigfit topursue Uamingin their ozonstyU 
as zueCtas a need to (earn thestytes of 
others. 

♦I* ^Everyone is an educational institution, has 
something to teach, and something to 
(earn. 

❖ Lift is ectectic. 9{p one way of teaching, 
teaming, or teadingfits ail situations. 

❖ "We don't succeed until ive altsucued. ^^^^ 
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iiJIi.. THE STUDENT IS ... 




... a critical and important member of the University family 

. . . working hard to develop his or her potential to its fullest. 

... a flesh and blood human being with feelings and emotions. 

. . . multifaceted - old, young, black, white, red, brown, yellow, male, 
female, married, single, liberal, conservative, rich, and poor. 

... the purpose of our work. 

Students pay us a compliment by trusting us to serve them. 

A message brought tc you by the Associated Students of Arizona State University 
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Cone of Learning / Another View 

A 



Wd Tend to 
Remember 



120% 



30% 



50% 



yo^/o 




I Reading \ 



" yhWaring Words ^ 



^Looking at Pictures""^ 



WatcNng a Mo^a 



Our Level of 
ln\'ofvement 



VatbaJ Racaiving 



LooMng at an Exhibit 
WatcNng a Demonstration 
Saaing ii Done on Location 

Partcipating in a Discusson 
Gving a Tak 



Visual 
Receiving 



Doing a Dramatic Presantatton 



SftTulatng tha Real Expeiienc* 



Doing the Real Thir>g 




Effectiveness of Learning Modes 




C^hsegi/ences of Teaching and 
"^Learning Style Mismatch 

^ (Felder, FIE Meeting, Nashville 1992) 

❖ Students: 

~ Become frustrated, bored and inattentive 

- Do poorly on tests 

- Get discouraged about tiie course, curriculum, themselves 
~ Ciiange to anotlier curriculum or drop out 

❖ Faculty: 

- Get defensive or liostile 

- Question tlieir clioice of profession; focus on research 

<* Society loses potentially excellent engineers: 

- visual, active, reflective learners (most students) 

- inductive learners and sensors (experimentalists, plant engineers) 

- global learners (systems thinkers, creative researchers) 



::iCiCc^®s^^ ... summary 

...... 

students may tend toward 

1 . Sensing or intuitive perception 

2. Visuai or verbai Input 

3. inductive or deductive organization 

4. Active or refiective processing 

5. Sequential or global understanding 

<• All combinations of types are needed in 
science and engineering 

❖ Most of our teaching is adsfracf (intuitive), 
verbal, deductive, and sequential. Students in 
our classes tend to be passive. 

•> To best serve all our students we should be 
reaching all types, not just one. 

❖ If we don't, students suffer, we suffer, and 
society loses valuable contributors. 



^^Exeerpts from a review of Richard J. 
%iglits's IHarvard Assessment Seminars 
(1990 and 1992 Reports)* 

❖ "...the single teaching technique that dominates both the 
1 990 and 1 992 reports is the use of small groups 

❖ Small group work was considered as especially effective 
for science majors and women 

❖ and as virtually essential for women in math and science." 



*Jim Cooper, Cooperative Learning and College Teaching, 
Volume 4, Number 1, Fall 1993, page 15 
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Sbme Cooperative Learning 
\Structures and Procedures (7) 
V (Bits & Pieces) 

❖ Think-Pair-Share or Formulate-Share-Listen-Create 

❖ Numbered Heads Together 

❖ Simple Jigsaw and Extended or Complex Jigsaw 

❖ Group Discussion with Talking Chips 

❖ Three Minute Essay 

❖ Structured Controversy 



X 



Cooperative Learning Bromides 
(Bits & Pieces) 



Contact Before Work 

(i.e., provided for some brief exchange between participants) 



Begin With the End in Mind 

(i.e., specify the objective or competencies of the experience) 



Need to Know Before Knowledge 

(i.e., develop an interest in or need to know the material, or 
competencies, to be realized from the experience) 



Structure Before Task 

(I.e.. communicate the structure or how the task is to be 
accomplished before commencing the task) 
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€(||^perative Learning Bromides 
(continued) 



Balance Process (the How) with Content (the What) 

(I.e., during the experience, balance the lime and focus on ttie 
process as well as on the Content/Task/Product) 

^heck for Understanding at Critical Points or Times 
(e.g., have someone paraphrase the structure, tasK, or conclusion 
beiore proceeding with the next step) 



Process Check 

(e.g., perform a process check at the end of the experience or 
at important Interim steps using "+delta", checklists or other 
forms to obtain timely feedback from the participants) 



/ Reflection 

f (i.e., pause for brief or even extended periods to think about what you have learned 
V and/or the process used to learn; keep an academic journal of your reflections 




Essential Elements 

of Active Learning (4) 



❖ positive Interdependence 

❖ individual Accountability 

❖ ^roup Processing 

❖ Social Skills 

❖ ^ace to Face Interaction 

"PIGS Face" 
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V ^ Simple Jigsaw Exercise 
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❖ Count off in your groups from 1 to 5 

<* Depending on your number you will read a 
paragraph (3 minutes) 

- 1's read about Positive Interrft^pendence 

- 2's read about Individual Accountability 

- 3's read about Group Processing 

- 4's read about Social Skills 

- 5's read about Face to Face Interaction 

❖ Prepare a 1 minute tutorial on your reading to 
teach your other group members what you 
have learned (5 minutes) 

<* In sequence deliver the tutorials (5 minutes) 
<* I will call, at random, for discussion of these 
paragraphs (5 minutec) 



\ Positive Interdependence 



Positive Interdependence exists when students believe that they are (inked 
with others in a way that one cannot succeed unless the other mennbers of 
the group succeed (and vice versa). 

Students are working together to get the job done. In other words, sti ' ients 
must perceive that they "sink or swim together." 

in a problem-solving session, positive interdependence is structured by 
group members 

(1 ) agreeing on the answer and solution strategies for each problem 
(goal interdependence) and 

(2) fulfilling assigned role responsibilities (role interdependence). 

Other ways of structuring positive interdependence include having common 
reward, being dependent on each other's resources, or a division of labor. 
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^ Individual Accountability 



Individual accountability/Persona! Responsibility requires the teacher to ensure that 
the performance of each individual student is assessed and the results given back 
to the group and the individual. 

The group needs to know who needs more assistance in completing the 
assignments and group members need to know they cannot "hitch-hike" on the 
work of others. 

Common ways to structure individual accountability include: 

❖ giving an individual exam to each student, 

❖ randomly calling on individual students to present their group's answer^ 

❖ giving an individual oral exam while monitoring group work (e.g., while the 
individual student is delivering an 'expert' tutorial to a small group). 



Group processing (e.g., process check) involves a group discussion of how well they 
are achieving their goals and how well they are maintaining effective working 
relationships among members. 

At the end of their working period the groups process their functioning by answering 
two questions: 

(1 ) What is something each member did that was helpful for the group and 

(2) What is something each member could do to make the group eve.i better 
tomorrow? 

Such processing: (1 ) enables learning groups to focus on group maintenance, (2) 
facilitates the learning of collaborative skills. (3) ensures that members receive 
feedback on their participation, and (4) reminds students to practice collaborative 
skills consistently. 




< 
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Sqpial Skills 




Social (collaborative) skills are necessary for effective group functioning. 

Students nriust have and use the needed leadership , decision-making, 
trust-building, comnriunicatlon, and confllct-nrianagenrient skills. 

These skills have to be taught just as purposefully and precisely as 
acadenriic skills. 

Many students have never worked cooperatively in learning situations 
and, therefore, lack the needed social skills for doing so. 



< 



Face to Face Interaction 




Face to face pronriotlve interaction exists among students when students 

orally explain to each other how to solve problems, 

discuss with each other the nature of the concepts and strategies being 
learned, 

teach their knowledge to classmates, and 

explain to each other the connections between present and past learning. 

This face to face Interaction is promotive in the sense that students help, assist, 
encourage , and support each other's efforts to learn. 
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[Session Agenda 



• Before Class Starts 

• find a group with less than four 
members and sit down with them 

• Contact Before Work 

• Getting Started 

• Assessment - Introduction 

• Levels of Learning Jigsaw 

• becoming an expert 

• break 

• delivering a tutorial 

• Introduction to Competency 
Matrices 



Pt^ratlon 



5min 
8min 
15min 

34 mln 
10 mln 
18 mln 

20 mln 



start 
Time 



iThe Environment and Purpose 



m 



Learning Environment 

• Active (as opposed to Passive) 

• Group or team based 

• Workshop facilitators (not Lecturers) 

Purpose and Expected Participant Outcomes 

• Become aware of what assessment entails 

• Become aware of how assessment is done 

• Become interested in the assessment process 
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i ' Focus on Facilitator' Signal 



The facilitator needs your attention: 

• Raise your hands to inform your neighbors 

• Finish your sentence 

• Do NOT finish your paragraph 

• Turn toward the Facilitator 



Issue Bin ( a useful tool ! ) 



Someone will be assigned to be the issue Bin 
Collector 

The following issuer will be assigned to the Issue Bin: 

• topics that will or may be addressed later 

• questions that can or should be deferred until the 
end of the workshop 

• items that can or should be the subject of future 
workshops 

Paraphrase the issue and record on the board or a 
piece of paper which is always visible 
At the conclusion of the session or workshop, the 
issues in the issue bin are brought out, one at a time, 
and discussed to see if they are still issues. 

Any issues which remain after the discussion must be 
addressed in a future workshop. 




Code Of Cooperation 



EVERY mQmbor Is responslbla for tha team's progress and success. 
Attend all team meetings and be on time. 
Come prepared. 

Carry out assignments on schedule. 

Listen to and show respect for the contributions of other members; be an 
active listener. 

CONSTRUCTIVELY criticize Ideas, not persons. 

Resolve conflicts construct ivaty. 

Pay attention, avoid disruptive behavior. 

Avoid disruptive side conversations. 

Only one person speaics at a time. 

Everyone participates, no one dominates. 

Be succinct, avoid long anecdotes and examples. 

No rank In the room. 

Respect those not present. 

Ask questions when you do not understand. 

Attend to your personal comfort needs at any time but minimize team 

disruption. 

HAVE FUN !!! 

*> 



I 

IWhat Do You Know? 



;:; • Do you know the symbol used for oxygen? 

• Do you know the valence for oxygen? 

I • Do you know the obit or spin direction of the 
valence electrons? 

• Can you balance the following chemical 
equation? 

O2 + H2 > H2O 

• Can you write down and explain the half 
reactions that are associated with the above 
reaction? 

: • Can you propose a process for the creation of 
methyl alcohol (methanol CH3OH)? 



31 



I What Is Your Attitude? 



How do you feel when I tell you that 
molecular oxygen in our atmosphere is O5? 

How do you feel when I tell you that the 
atomic weight in amu's of oxygen is 159, 15.9, 
15.99, 15.999, 15.9993, 15.9994, 15.9995? 

How do you feel when I tell you that the 
valence electrons for oxygen are in the 2p 
orbital? 

How do you feel when I tell you that oxygen 
can be generated from moon rocks? 

How do you feel when I tell you that a new 
atomic element has been found in the last six 
months? 



Levels of Learning (LoL) & 
Degrees of Internalization (Pol) 



i 
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Cognitive Aspects (thinking aspects)^ 

• Knowledge 

• Comprehension 

• Application 

• Analysis 

• Synthesis 

• Appreciation (Evaluation) 

Affective Aspects (feeling aspects)^ 

• Receiving 

• Responding 

• V3^">"g iBIoomet 811956 

2Krathwohl et a! 1964 
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LoL & Dol Jigsaw 




Class 



Jigsaw 
Teams 



■ 1 Count off by X 
2 Local* Exp«n uU* 
for your numb«r 



Expert 
...Groups 

: Movo to E]qp«t : 
< T«b<* 



: Exp«rt Groups j 
-^.i La«rn Their 
: Mstwtal 



i: 



] Exp«rt Group* 
I MouM Tutorial 



1 Count oH 
: 2. Locat*Jig$«wub(« 
for your rKimbar 



Non-«xp«rts 



o- 



Pr«par«& 
Gv«R*port 



Expert Reading 



Everyone review Part I of the Self Evaluation 
Guide (Section C of Workbook) 

In addition, read from Appendix A of the 
Guide as follows: 

• Table 1 is to read Valuing & Knowledge 

• Table 2 is to read Receiving & Comprehension 

• Table 3 is to read Responding* & Application 

• Table 4 is to read Valuing & Analysis 

• Table 5 is to read Receiving & Synthesis 

• Table 6 is to read Responding & Evaluation 

If you have people who do not have the Orange Workbook, 

please share your workbook for this 

exercise. 
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Becoming An Expert 



Read the material that has been assigned to your experf table 
Discuss the reading with your expe/t team members to reach 
understanding on meaning and importance of the various 
ideas 

As a team, develop a two minute tutorial that you can use to 
teach other members of the class about your level or degree 
As an aid in preparing your tablets tutorial prepare a class 
assignment that, if done, could be used to demonstrate 
mastery at your table's LoL. What sort of work product would 
be created by a person doing your assignment at your level? 
• if your group has become experts in either Knowledge, 
Comprehension or Application prepare an assignment on: 



• if your group has become experts in either Analysis, 
Synthesis o** Evaluation prepare an assignment on: 
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Presenting The Tutorial 



• The two minute tutorials should be delivered in the following 
sequence: 

• Receiving 

• Knowledge 

• Responding 

• Comrjrehenslon 

• Application 

• Analysis 

• Valuing (delivered by either fifth team member or person who did not do 
Analysis) 

• It is a good idea to check each other to ensure that everyone is 
learning the material to at least the Knowledge LoL and 
Receiving Dol 

• After all the tutorials have been delivered spend a few 
moments in clarification 
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34 



I 
I 
I 
I 

I 
I 



::::x::.:.:.K.x.:vK 



IWhat is Assessment? 



Assessment is a process used to answer the 
questions: 

• what do you know ? 

• how well do you know It ? 

The answers to these questions are based on 
the evidence^ that you supply to the person 
doing the assessing 



^ In this class all the evidence will be collected and organized 
in a Portfolio (see Part III of Self Assessment Guide in Section 
C of Orange Workbook for more information on Portfolios) 
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Stages of Knowledge^ 



^ you know 


3 you know 


that you do not know 


that you know 


(conscious incompetence) 


(conscious competence) 


1 you do not know 


4 you do not know 


that you do not know 


that you know 


(unconscious incompetence) 


(unconscious competence) 


Low 


High 



High 



Low 



Competence 
' Mr. House - Hewlett Packard 
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Jhe Workbook Competency 
S Matrix 



Open your Orange Workbooks to Section H and turn to page 2 of 5. 
For each of the following questions silently consider an answer and 
then share with your group members. A person selected at random 
will be asked to give an answer. 

What are some of the topics to be covered by the material 
in the workbook? 

How have the topics been organized on a page (you might 

like to !ook at the Tree Diagram on page G - 1 9} 

Is it assumed that you have any pre-knowledge about any 

of these topics before you came to class? 
. What do the 2's, 3's, 4's, and 5's mean? 
. What level of cognitive and affective behavior Is expected 

afterthis class for 

• Knowledge 

• Synthesis 

• Reflection Logs 



— lUpdating Your Workbook 
1 I Competency Matrix 



• Look at each of the competencies 

• if you recognize that a competency was presented today 
put a check in the appropriate row under Receiving 

• for each competency that you checked for Receiving, find 
the slide in the workbook that Introduced the competency 
and enter that slide number In the appropriate row under 
Knowledge. 

• Find the competency for the items related to 
your tutorial 

• put a check in the appropriate row under Responding 

• for each competency that you checked for Responding, 
find the slide in the workbook that you used In 
developing your tutorial and enter that slide number in 
the appropriate row under Comprehension 
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Preface 



In this class you will be crearing a Portfolio of your work and developing a special index to this \vork. 
A Portfolio is just a fancy name for an organized collection of all the work you will underta:. j during 
the semester. Some of you may already have methods for organizing your work; however, in this 
class you will be asked to organize the work in a specific manner which will allow a simplified means 
of assessing your performance. At the end of the semester your Portfolio will contain the evidence 
of what you did and, by implication, evidence of what you have learned. Theoretically, it should be 
possible at the end of the semester for anyone who knows the course expectations to look through 
your collection of work and decide how well you did in meeting those course expectations (i.e., assess 
your performance). The grade you receive in the course is a function of this type of assessment. 

The assessment job can be simplified if it is easy to find the specific work that is related to a specific 
course expectation. For example, if during the course it is expected that you will learn how to extract 
derivatives of trigonometric functions, it would help to know where in your Portfolio there is work 
related to extracting trigonometric derivatives. This connection from expectations to specific pieces of 
work can be constructed in several ways; however, in this class the connection will be created using 
an index known as a Competency Matrix. You will build this index continuously during the semester 
as you do your course work and put it into your Portfolio. 

You may be asking yourself Why all the fuss with Portfolios and Matrices? What was wrong with (he 
old way of just submitting the work and getting a grade? Using Portfolios and a Competency Matrix 
greatiy enhances the assessment process: first, assessment is more uniform (i.e., fairer); second, 
individuals other than the course instructor may do the assessing. In particular, it makes it possible for 
you to do the assessment. Exactly how much self assessment you actually do will be a function of the 
specific course and faculty; however, you can expect to do some self assessment, and in some cases 
you may do the majority of assessment. 

Acknowledgments 

The material presented in the Guide is a compilation of the thoughts and experiences of Drs. Jim 
Bailey, Lynn Bellamy, Don Evans, Mark Henderson, Darwyn Under, Barry McNeill, Jack Mster, 
and Greg Raupp as they attempted to utilize the concept of a competency matrix in the evaluation of 
student periforaiance in their classes during the last several years. 
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Part I - Introduction 

Have you ever wondered: 

1 . what an education is, or 

2. what it means when you learn something, or 

3. if you only learn in school, or 

4. whether your education stops when you leave school? 

^These are not easy questions to answer and some people spend their entire lives attempting to 
formulate answers; however, difficult as it may be, you do need to think about answers to these 
questions. Before reading further you might like to jot down some reflections on these questions. 

Education and Learning 

So what are some answers to these questions. Starting with the most difficult, i.e., question 1, 
education is defined in The Random House College Dictionary as: 

the act or process of imparting or acquiring general knowledge and of developing the powers 

of reasoning and judgment. 
This shows education to be an active, dynamic thing {imparting, acquiring, developing). Education 
causes a change in your general knowledge and/or in your ability to reason and to use judgment. 

Following this definition it seems reasonable to expect learning to be associated with the actual 
acquiring or developing part of this definition. When you learn a history lesson you are obtaining 
knowledge about the activities going on during some period of time. When you learn to drive a car 
you gather knowledge about the car, the rules of the road, and develop a sense of when it is and is not 
safe to pass, a reasoning/judgment activity. So learning is the method or way by which you acquire 
and develop knowledge and wisdom. There is no single, correct, way to leam something; different 
people leam things in different ways. 

The third question, suggesting that you can only learn something in school, is clearly erroneous. 
School in only one of the many places that you leam. You certainly learned how to walk and talk 
outside of school. You learned to drive a car outside of school. In fact, much of your learning takes 
place outside of school. Education is not limited to schools or universities. 

Life Long Learning 

Now to the last and probably most important of the four questions: does the end of schooling mean the 
end of education? The answer is a resounding NO. In fact, your education never stops. If you want 
to continue to be a productive and useful person you must continue to leam throughout your life. As 
your interests and needs mature and change you need to acquire and develop new information and 
judgmental skills. A university degree merely starts you down the road of life long learning. 

If the University does not complete this education process, what does it do? Universities help you 
move to adulthood with its myriad of responsibilities. At the undergraduate level. Universities are 
learning environments that help you learn how to learn so you, as an adult, can take full responsibihty 
for your own education. Universities help you to learn^ at a time when you may not be mature enough 
or have enough knowledge to know what you need to leam. However, Universities must eventually 
wean you from depending on authority figures to define what you need to leam; eventually you will 
leave the University and unless you know how to leam, your education will stop prematurely at 
graduation. 



^e.g., a structured curriculum 
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Introduction 



The primary goal of a University is to graduate Life Long Learners, people who can learn by 
themselves, people who can learn and want to learn autonomously. Philip Candy has developed a set 
of characteristics of an Autonomous Learner (Figure 1). Read through Figure 1 and see how many of 
these traits apply to you. Do not be overly concerned if the answer is not many; much of what you 
will do in this class is aimed at starting the change in your attitude toward learning; i.e., moving you 
closer to the profile of the autonomous learner. 

Educational States 

By definition, education is a process, which means that things change. So what is changing and how 
do you measure this changing quantity? What is changing are your behaviors, attitudes, knowledge, 
skills, and reasoning powers. This collection of attributes is covered under the umbrella concept of an 
Educational State. Thus, the definition of education becomes: 

Education is the process that changes a person's educational state 

Traditionally, the evaluation of the status of your educational state has been done by course instructors 
who assign a grade to indicate your state. While it may be relatively easy to let instructors do this 
evaluation of your work (i.e., you do the assigned work, submit the work, and wait to see what the 
instructor reports), in the long term you must be the person doing the evaluation. You cannot 
continually rely on someone else to tell you how you are doing; you must learn how to evaluate your 
own educational state so you can make the required improvements. 

How can educational states be defined or characterized? While there are many ways to do this, one 
way is to characterize your educational state by the activities and actions (i.e., behaviors) of you and 
your teacher. Reflect back over your time in school; you should be able to recognize the major 
changes that have taken place in your activities. During the first several years you learned facts and 
worked simple, single concept problems; during the latter years you worked problems that 
combined many different concepts and skills. 

In the early 1950's a group of educational psychologists addressed the problem of defining educational 
states. To quote from the Foreword to their first efforts: 

It (this work) is intended to provide for classification of the goals of our educational system. It 
is expected to be of general help to all teachers, administrators, professional specialists, and 
research workers who deal with curricular and evaluation problems. It is especially intended to 
help them discuss these problems with greater precision.^- 

These psychologists divided the problem into three behavioral domains: the cognitive, dealing with the 
recall or recognition of knowledge and the development of intellectual abilities and skills; the affective, 
dealing with interests, attitudes, appreciations, values, and emotional sets or biases and the 
psychomotor, dealing with the manipulative or motor-skill area. These psychologists attempted to 
define a finite set of recognizably distinct behaviors and then sequence these behaviors in the order 
they occur when someone is learning (i.e., they were striving to define a taxonomy of cognitive, 
affective, and psychomotor behaviors associated with learning). 

Cognitive Domain 

A taxonomy^ (a handbook), that dealt with the cognitive domain, was published in the mid 1950's. 
After much discussion and testing of ideas with a wide range of colleagues. Bloom et al. proposed a 
cognitive taxonomy containing six major categories. In order of increasing complexity (and learning) 
they were: Knowledge, Comprehension, Application, Analysis, Synthesis, and Evaluation. Each of 
these categories was characterized by a different set of abilities (behaviors) exhibited by a person 
operating in the category. This taxonomy was widely accepted and is the basis for many curriculum 
development efforts. 



^Taxonomy of Educational Objectives Book 1 Cognitive Domain, Bloom et al, Longman, 1956. 
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Learners Capable of Autonomous Learning Will Characteristically: 

Be Methodical / Disciplined 

• Be able to focus on an area of interest 

• Develop individual plans for achieving goals 

• Establish personal priorities 

Pay close attention to details of an ongoing project 
Be Reflective / Self-Aware 

• Decide what knowledge and skills to learn 
Have a self-concept as an effectivf learner 
Know his or her strengths and weaknesses 

• Understand his or her own values, interests, abilities, and knowledge 
Demonstrate Curiosity / Openness / Motivation 

• Be curious, with a continual need to leam 

Be "cogniiively open" with regard to phenomena 
Confront questions and problems willingly 
Be Flexible 

Be able to leam in many situations - from conversations, through reading, and by observation 
Be able to leam from listening, taking notes, reading, or memorizing 

• Be able to accept or reject material 
Be able to achieve or abandon goals 

Be Persistent / Responsible 

Be capable of intellectual concentration 

• Stick to plans, modifying as necessary 

• Have a tolerance for frustration 

Detect and cope with personal and situational blocks to learning 
Have Developed Information Seeking and Retrieval Skills 
Intelligently select and use most relevant sources of information 

Identify, and know how to use. resources appropriate to different kinds of learning objectives 

• Be able to establish feedback mechanisms for day-to-day performance 
Have Knowledge About and Skill in "Learning Processes" 

Be capable of reporting what he or she has learned in a variety of ways 
Be able to decode a message - textual, auditory, or visual 
Have developed skills in taking notes, remembering, and relating 
Develop and Use criteria for Evaluating 

Be able to select what is of value from the mass of information available 
Participate in diagnosing prescribing, and evaluating his o r her own progress 

Figure 1 Profile of the Autonomous Learner^ 



Excerpted from : Philip A. Candy, Self-Direction for Lifelong Learning. San Francisco. Jossey-Bass, 1991. pp 459 - 
466 
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In the late 1980's and early 1990*s David Langford, in attempting to implement an important aspect of 
the quality culture (empowerment) into the classroom, recognized that the cognitive taxonomy could be 
used by the students as well as the teachers to determine where they were relative to these various 
objectives. Langford proposed having the students use these educational objectives to do self 
evaluation. Lan^ord renamed the objectives Levels of Learning, changed the name of Comprehension 
to Know - How, and developed summaries of the types of activities a student and teacher would 
undertake when they, the students, were operating at these various Levels of Learning^. 

A modification to Langford' s material on typical activities can be found in the first six pages of 
Appendix A. This material consists of answers to a set of seven standard questions. The answers to 
the questions change as the Level of Learning changes. For example, at the Knowledge Level the 
question How do I know I have reached this level? is answered , / recall information; at the Synthesis 
Level the same questions is answered, / have the ability to put together parts and elements into a 
unified organization or whole that requires original, creative thinking,. Carefully read the answers to 
these questions in Appendix A, paying especially close attention to the first two and last three 
questions for each of the levels. 

Affective Domain 

As the psychologists had pointed out in the 1950's, there is more to defining an educational state than 
is covered by cognitive behaviors (Levels of Learning). There are the affective behaviors that must 
also be considered. These affective behaviors cover a wide variety of issues ranging from the 
willingness to receive and try new material, or the interest in what is being learned, to a sense of value 
with regard to what is being learned or how the material is being learned. Aside from these sorts of 
issues, the affective domain also involves the related issue of Character (honesty, integrity, 
truthfulness, etc.). Myron Tribus, in several of his recent essays^, states that Character is one of the 
major categories of attributes that should be cultivated in a school or university. 

What are the affective stages you pass through during the learning process? In the mid 1960's a subset 
of the group of educational psychologists who had produced the cognitive taxonomy published a 
second taxonomy^, one that dealt with the affective domain. Compared to cognitive behavior, 
affective behavior is a much 'softer' (i.e., harder to define) type of behavior with the result that the 
development of the affective taxonomy was a rather difficult task. It was hard to define exact 
categories of behavior that had definable boundaries; the categories seemed fuzzy and blurred rather 
than crisp and focused, as was the case for the cognitive categories. Eventually Krathwohl et al. 
defined five major categories which, in order of increasing complexity .called Degrees of 
Internalization) were: Receiving, Responding, Valuing, Organization, and Characterization by a Value 
or Value Complex. Each of these categories was characterized by the set of behaviors exhibited by a 
person operating in the category. Because the affective domain is rather fuzzy, the affective taxonomy 
has never enjoyed the wide spread acceptance given to the cognitive taxonomy. 

The last pages of the material in Appendix A address the affective domain. You will find general 
material for the first three Degrees of Internalization. Only the first three Degrees of Internalization are 
discussed since the final two degrees are not considered by the author to be appropriate for use in the 
university classroom setting. These top two degrees involve the integration of the value of learning 
and material learned into your personal value structure, a structure that encompasses much more than 
educational values; thus movement to these higher degrees of affective behavior must take place 
outside the classroom, in the broader context of your life. 



^Total Quality Learning Handbook, Langford Quality Education, 1992 

^Quality Management in Education, and Total Quality Management in Schools of Business and of Engineering, Myron 
Tribus, Exergy, Inc., Hayward, CA 

^Taxonomy of Educational Objectives Book 2 Affective Domain, David Krathwohl ct al, Longman, 1964. 
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Psychomotor Domain 

The last piece of the educational state is defined by your psychomotor skills and abilities. As of this 
writing there has been no definitive taxonomy developed for these attributes; therefore, you will not be 
expected to evaluate this part of your educational state. 

This Guide 

The remainder of the material in this guide is organized into three parts and a set of appendices. Part II 
discuss expectations concerning the presentation of your work. Part III introduces the mechanics of 
creating and using the portfolio. Part IV is a more detailed discussion of the assessment process and 
attempts to explain the steps in more depth. If you have never used a Portfolio or Competency Matrix 
you will want to read the first part of Pan II and all of Part III, and do the work suggested. Once you 
have a general understanding of what is expected you will want to read Part FV to get a better 
understanding of the process and improve your process. 
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Part II Presentation of Technical Work 

Before discussing how to organize and assess your work it is necessary to generate and present some 
work You have, of course, had experience in this area^; however, I am sure you have had situanons 
where the evaluation of your submitted work was lower than expected in part because the work while 
technically correct, was not presented well (in rhe eyes of the teacher) Your ability, and that of 
others to assess your work depends, to a large extent, on how you choose to present (i.e., document) 
your work. A large part of the evaluation process is directly related to the quahty of the presenmtion 
which is related to the evaluator's expectations. It is necessary that you realize what is expected so that 
you can supply it. 

This portion of the guide presents some general traits of quality presentations and then addresses some 
specific presentation issues. 

General Presentation Traits 

While specific expectations for presentations vary depending on what sort of task you are documenting 
or who IS reading (evaluating) the material, there are some general, philosophical guidelines, that can 
helD vou achieve a good presentation (documentation) of your work. Initially these ffaits may seem to 
require extra work Sid be solely associated with a desire to make the work understandable to someone 
other than yourself. Upon reflection, however, you will realize that this work is not really extra. 
When vou do the extra work required to make the work understandable (i.e presentable) to other 
neoDle vou actually achieve a better understanding of what you have done (are doing) and with this 
understanding comes the knowledge of how to do the work even better than you first envisioned. It is 
hard to consider anything that allows you to do your work better to be considered as extra work. 
When presenting your technical work, here are some traits you should strive to achieve: 

1. explain what is going on 

You should attempt to let the reader know, at all times, what is going on. This nieans 
explaining what is being attempted, why it is being attempted, and what method is being 
used. This also means that at the conclusion of a task you explain the consequences or 
meaning of what you have just done. 

2. make explanations clear 

Trait #1 addresses the need to explain what is going on; this second trait involves making 
explanations clear (i.e., reducing the variation in interpretation ). Clanty is often enhanced 
by using sketches, plots, figures, or some other type of graphical presentaaon to augment the 
text. Clarity is also improved when terms and variables are defined. 

3. make it readable 

If vou make the work easier to read you will improve the presentation. Explaining clearly 
what is going on improves readability. Other ways to improve readability include providing 
neat work, clean sketches, binding the material so that it does not fall apart and yet is easy to 
access, numbering the pages, etc. 

4. make it clear whose work it is 

It should always be clear who did the work and when the work was done. This is usually 

done by dating and initialing each sheet of the work product. 
You should strive to incorporate these traits into all your presentation efforts; omitting even one of the 
traits reduces your ability to produce an adequate presentation. 



buring high school and college you have done homework, generated reports, etc. that were submitted to a teacher for 
evaluation (grading). 
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Presentation Expectations for Some Types of Technical Work 

There are clearly many different types of work to be documented and no attempt will be made here to 
address all these types. Rather, the expectations for a few types of work, work that you will encounter 
3I!'.'tf ^-n f classes, will be discussed. Homework, plots, computer models, and other work 
products will be discussed. If this is your first time through the guide you may want to skio the 
matenal on models and come back to it at a later time. 

Homework 

You have homework in every engineering class that you take. While you will find that the format for 
homework (e.g., the size of the margins, the order in which material is presented, the way results are 
presented, etc.) will vary fi-om class to class, you will also find that the expectations for what is 
presented will not vary significantly fi-om class to class. 

When the general traits of good presentation are applied to your homework, the following set of 
expectations emerge. ^ 

1. State The Problem 

Write down the problem statement as it is given in the book or assignment sheet This sets 
the context for the problem and the work (addresses general trait #1). 

2. Draw An Appropriate Diagram Or Sketch 

The diagram should show the information given in the problem and show what is required or 
desired, l hus, if the problem were to determine how much force it would take to lift a safe 
using a lever and fulcrum (pivot), then your diagram would show the lever, the fulcrum the 
sate, and you pushing down on the lever (addresses general trait #2). 

3 . Describe Your Process of Analysis 

Describe in a few sentences what process, methods, and approaches you used in analyzing 
the problem. You should include any assumptions you have made. For example, in the 
sate/lever problem, you might report that you are going to use a force balance (or 
equilibrium). This should be more detailed than just stating a general name (e.g., force 
balance); you should describe a few of the key steps in the named process (addresses general 

tTH.ll Tr^fc ), 

4. Perform the Analysis 

For the process described in 3 above, work the problem. Working the problem includes 
documentation of your analysis, including all appropriate intermediate results. Note that 
documentation of the analysis process (working the problem) is as important as the final 
result (sometimes known as the answer) obtained fi-om the analysis. 

5. Discuss the Results of Your Analysis 

You must contemplate or reflect on your results and present some comments about the 
results. What do the results mean? Is the result bigger (smaller) than expected? Is the resuh 
reasonable? What might be done next? These are only a few of the possible questions you 
could address when you are writing your discussion of the result (addresses general trait #1). 

Jfr Srw^^f • ^'l^'^"^ ^ '^^^^ °^ ^'g"'^' ^h^" "^"st discuss each of the plots 

or tables. What is the trend? Can you extrapolate? 

When you prepare your homework for this or any other engineering class, you are expected to 
adhere to these five points. Failure to follow anyone of these points means that you S^e not miting 
the expectations of most faculty (and as you will see later, not meeting your own presentation 
expectation s} . 
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Presentation of Technical Work 

Graphical Material (including plots) 

A considerable amount of information is contained in pictures; hence, your work will often contain 
significant amounts of graphic information (general trait #2). This graphic material is generally either 
pictures of an artifact or plots showing the relationships among some variables. Because pictures 
contain so much information concisely (i.e., pictures capture the essence of the material) graphic 
material is frequently copied and distributed, often without the accompanying text. Since this practice 
is so ubiquitous, it is critical, when preparing graphic material, that you: 

1 . add enough annotation to the material that the picture makes sense standing alone, and 

2. prepare the graphical material so that it can be reproduced. 

This means that you must include titles and labels on all your graphic material, and you do not put 
your drawings on green grid graph paper using a number 3 pencil. 

Aside from these two general items, there are some standard expectations concerning the presentation 
of plots that you will want to be aware of and follow. These plot expectations are: 

1 . Both axes of a plot must be labeled, including units when appropriate. The use of only a variable 
(e.g., V) as the label is discouraged because of the different interpretations (e.g., is V volume, 
velocity, etc.). Using only variables as labels is not in keeping with general trait #2. 

2. The divisions for each axis must be marked. The value the plot origin (intersection point of axes) 
must be clear. 

3. The dependent variable (the variable calculated) is always plotted on the vertical (y) axis 
while the independent variable (the variable you changed) is plotted on the horizontal (x) axis. 
This expectation has ramifications concerning plot titles (see item 4). 

4. The plot must have a descriptive title, including a Plot or Figure number. The title PvsT is not 
considered to be descriptive and must be replaced with something like Container Pressure as a 
Function of the Internal Tank Temperature. The title should list the dependent variable first 
and the independent variable second. For example, if the title is Production Flow rate vs 
Compressor Discharge Pressure, you know that the discharge pressure (mentioned second) is the 
independent variable and is shown along the horizontal axis while the flow rate (mentioned first) is 
the dependent variable, plotted along the vertical axis. 

5. If you are displaying a number of related curves on the same plot you need to have a concise 
legend defining the various curves. 

6. Data points used in generating the plot are generally shown. 

7 . Smooth curves are generally dra^vn through or near the data points to show the trend. 

The above expectations are true for any plot, whether it is hand or computer drawn, found in your 
homework or in a report. You may find that computer drawn plots do not meet some of these 
annotation standards. In such cases it is entirely permissible to add the annotation after the plot has 
been generated. You can add this annotation by hand, with a typewriter, or transfers. 

Analytical Models^ 

Models play a significant role in engineering and it is very important that you realize what is expected 
when presenting your work. The material that follows is divided into three major areas: first, how to 
present the work related to the development of the model; second, what is expected concerning the 
model itself; third, presenting the work related to the use of the model. 



^This material makes sense only after you have some idea of what analytical models are. If you do not have this 
knowledge then skip this section and come back later after you start building and using analytical models. 
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analytical model development 

What is expected for analytical model development follows directly from the general traits. 
Analytical model development concerns all aspects of assembling the set of equations that are used to 
predict performance. After reading the model development material it is expected that the reader will 
believe that the work is complete and correct. It is expected that all model development work will: 

1 . define the system being modeled (general trait #1 & 2), 

2. define the model limitations (general trait # 1), 

3 . define the model variables (general trait #2), 

4. establish the appropriateness of the modeling method (general trait #1 ), and 

5. establish the correctness of the model's results (general trait #1). 

The first three items are relatively straight forward and probably reflect the sorts of definitions you 
have already been including in your presentations. However, the fourth item may not currently be part 
of your presentations. For homework problems the appropriateness of the modeling method is often 
assumed (implicit and not stated); why else would the assignment have been given. Even so, it is 
important to address this issue when presenting model development. There are several ways to 
establish the appropriateness of your model. You can cite experts. You can also construct the model 
from basic principles of science and mathematics. Appropriateness also requires addressing whether 
you have the necessary resources (time, money, computer power, expertise, etc.) to implement the 
model; i.e., applying Occam's razor. 

The last item is an issue you may or may not have explicitly addressed to date. It is expected that you 
will present work to show that the model is actually working correctly (i.e., that there are no errors in 
the model). This is not a trivial task and there are a variety ways you can show that your model is 
working correctly, each with a different degree of credibility. Probably the best way to show model 
correctness is to run" your model with a test case that has known results (e.g., mn a homework 
problem from a text book). If this is not possible the next best method would be to show that the 
model generates results (designs) similar to existing designs (e.g., the models predict rocket nozzle 
sizes of the same size as those found in industry). Next might be to show that the model behaves 
correctly (i.e., when inputs are changed the outputs change in the predicted fashion). Some 
confidence in model correctness can be obtained by showing that the model generates correct order of 
magnitude answers. Probably the method least likely to convince, although it does provide some 
evidence of model correctness, is to show that the model reproduces the results of some earlier paper 
and pencil work. 

computer models 

The above discussion applies to any model that is developed, independent of the method or tools used 
in running the model. When developing computer models there are some additional documentation 
expectations that help assure that the computer model is understandable and correct. 

high level languages 

If the model is developed using a high level language such as C, the computer code must include 
comments and a variable dictionary. A listing of the program must be included. The listing should be 
easy to read (i.e., not bound into the documentation in such a manner as to make reading impossible). 
Any model that exceeds several hundred lines of code should be broken up into a set of smaller 
subroutines or procedures with an accompanying flowchart to explain the general program logic and 
control. 



Version 3 



Workbook Section C 



Presentation of Technical Work 



equation solvers 

If the model is developed using an equation solving program such as TK then the documentation must 
include a completed Variable Sheet^ including the Units and Comments Columjis. The Variable Sheet 
must show a consistent set of values for all variables. You must also supply a copy of the Rule 
(Equation) Sheet. The Rule Sheet should contain some general comments to tie major sets of 
equations together. There must be enough annotation on the Rule Sheet to explain the basis for the 
various equations or sets of related equations. If the model uses any special user defined functions 
these should also be included in the material submitted. User functions should be documented in the 
same fashion as the equations on the Rule Sheet 

spreadsheets 

Documenting a model using a ^'preadsheet may be more difficult than using high level languages or 
equation solvers because spreadsheet equations use cell addresses rather than variable names. This 
problem has been partially ameliorated by including the ability to name single cells in a spreadsheet 
Names assigned to single cells may be used in the formulas in place of absolute cell addresses. Arrays 
may also be assigned names but individual elements of the array may not be addressed with the 
name. However, the array name may be used in built-in functions in place of the range. In 
spreadsheets, a clearly defined tide or heading must be assigned to each column and/or row of the 
sheet and it may be appropriate to use an adjacent cell to identify or label the contents of an individual 
cell. If the equations in the spreadsheet are not intuitively obvious, then you should append a report 
that explains what calculation is being performed in each cell. The repon may be appended using the 
'cell notes' feature of most spreadsheets. 

In developing a spreadsheet model, there are some matters to consider which will generally lead to a 
robust, easily maintained, quality model. In particular: 

1 . Start with a plan. Sketch out what the spreadsheet is going to look like on paper. 

2. Keep data entry and calculation areas separate. If the sheet has numerous inputs or data, you may 
want to have an entire screen reserved for the inputs or data. 

3. Use the paper sketch above to construct the actual spreadsheet. 

4. Concentrate the entry of your inputs or data in one area or block. Arrange the inputs or data in 
contiguous cells (i.e^, sequential rows in a few columns, or sequential columns in a few rows). 

5. Do not use numerical values for "design constants or parameters" in your formulae. Use cell 
references and assign names to the cells containing the current numerical values of these ''design 
constants or parameters". This will allow you to change these values without changing the 
formulae. For example, enter the numerical value for the time step in a cell and use this cell 
reference or name in your formula for calculating time. 

6. Separate, decompose, or parse large or complex calculations into a set of smaller or simpler 
calculations. Place the smaller or simpler calculations in adjacent or contiguous cells in the sheet. 

7. Always test, validate, or calibrate your spread sheet (see item 5 under model development). 

8 . Use cell protection for protection and highlighting of areas that contain formulae or other 
information that will not be changed frequentiy. 

9. Use column and row labels as well as cell labels extensively. Include operating instructions for 
the sheet (text boxes may be used for these instructions). In summary, clearly and completely 
annotate your spreadsheet model. 

10. Use the cell and sheet formatting features found in most modem spreadsheets to improve the 
appearance and presentation of your model. 



^Equation solving programs have a variety of input and output screens which they call sheets. Some of these programs 
can solve multiple cases by running in what is called List Solving. 



Version 3 



51 



Workbook Section C 



Presentation of Technical Work 



1 1 . Back up your files FREQUENTLY! And maintain at least two copies of the files on separate 
diskettes! This may feel like a tedious, unnecessary, or even unproductive task but the first time 
your files are damaged or lost you will value this task immediately! 

the use of models 

Once a model has been developed and validated (or calibrated, or shown to be correct, or tested, etc.), 
you will be using the model to assist you in understanding some system or making decisions based on 
your model. You may use the model one or two times, or in some instances, you may use the model a 
great many times (i.e., you will execute the model for a number of design cases). It is easy to be 
overwhelmed by the enormity of data that can be generated widi a computer model, and the 
presentation of this work needs careful consideration; in particular, how to display the results of all 
these runs and how to discuss the results of the runs. 

displaying the results 

The important results of any case, such as those used to substantiate a conclusion, can be displayed 
using plots and or tables. Tables are generally required for any plot shown; however, there is no need 
to generate plots for all the tabular data; only the most important relationships need to be plotted. The 
data from all cases that are run using the same model (i.e., the equations are not changed from run to 
run; only the input data are changed) should be grouped together in some logical order (chronological, 
variable changed from case to case, etc.). 

It is important that the numerical values of all the variables be known for each case or run. Generally 
you will have a large set of result variables that are not changed for any of the cases and a smaller set 
of variables tiiat are changed from case to case or run to run. A listing of the variables which are held 
constant during a case or run, and their numerical values, needs to precede the case results. For each 
case that you run, you will generate a set of lists, probably collected into a table, showing how the 
result variables changed. For each case, you must annotate the lists or tables of results with an 
explanation of what has changed from the previous case (i.e., specify the numerical values of the 
variables that changed for the each case). If you use a general table containing numerous result 
variables, only some of which are of interest for each case, you must identify or mark the variables of 
interest (using a highlighting marker is acceptable or you may use formatting in a spreadsheet). 

discussing the results 

When you present the results of your work you need to explicitly discuss the results (i.e., never 
submit a plot or a table without some discussion of the table or plot). This written discussion will 
probably precede the results (the tables or plots). When you develop this discussion you should look 
at the data and then tell the reader what you want the reader to notice (explicit presentation); do not 
make the reader figure out (i.e., infer) what is important or interesting about the data (implicit 
presentation). Explain what decisions you have made or will make based on the data. If the data has 
any anomalies do not ignore them; either attempt to explain why they exist or admit that after 
consideration you cannot really explain the unusual (unexpected) shape. The discussion must refer to 
the plots and tables by their names or numbers. 
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Now that you have work to present how should it be organized? Organizing your work is always 
important. Organized work is work that can be used again. It can be used to review the semester's 
work; it can be used to remind you how to do something a year later, it can be used to show someone 
else what you have done. There are many ways to organize the work (e.g., chronologically, by 
assignment number, alphabetically, etc.), each appropriate depending on the reason for organizing the 
work. In this class the goal is to organize the work in a fashion that allows ease of assessment. 

This is a three step process. First, an organizing structure must be defined and implemented; second, 
the work you do must be inserted within the created structure; and third, the index to the work must be 
updated. The first step is the creation of an empty, but functional. Portfolio; the second step is the 
filling in of the Portfolio; the third step is the updating of the Competency Matrix, The last two steps 
are the assessment steps and are the most difficult to do. This part of the guide is meant to make it 
possible for you to start the assessment process without completely understanding the process. Once 
you have a general idea of what is expected here read Part IV, which focuses on the assessment 
process itself. When you read Part IV you will begin the understand more fully the mechanical steps 
you have been asked to perform in this section. 

Creating an Empty Portfolio Structure 

The following steps are the process of setting up a Portfolio for a class. An example, using the 
Competency Matrix shown in Appendix B, is given. 

1 . Purchase a three ring binder with hard covers that can take standard three hole punched paper The 
size binder you need will depend on how much work you will be doing during the semester, unless 
you are instructed differently, a three inch thick binder should be adequate. You should also 
purchase a number of tabbed page dividers (see item 3 to determine how many dividers you need). 

2. Obtain the Competency Matrix for the class from your course instructor (see Appendix B for 
matrix to be used in the example). 

3 . Label your page dividers as follows and put them into the binder. 

1 . Mark the tab on your first divider 0-competency matrix. 

2. You now need to create a series of numbered dividers, one for each of the Learning 
Outcomes in your matrix. 

a. Review the Competency Matrix; note that the left hand column is labeled as either Learning 
Outcomes or Competency Category. If you note this column on each page of the matrix 
you will see from one to three (four) different topics listed. Each topic is a learning 
outcome (competency category) for the course. The matrix shown in Appendix B includes 
two learning outcomes: differentiation and team meetings. 

b. Review the entire Competency Matrix and count the number of different learning outcomes. 
If the number is N then use N more page dividers. For the Appendix B course you would 
get out two more dividers (i.e., N=2). 

c. Label each of your N divider's tabs with a number and Learning Outcome. For the 
Appendix B course you would have two dividers with tabs labeled as 
1-differentiation and 2-team meetings. 

3. Label the next divider N+1 class notes (for example it would be 3-class notes) 

4. Label the next divider N+2 work logs (for example it v>'ould be 4- work logs) 

5. Label the next divider N+3 run charts (for example it would be 5-run charts) 

6. Label the next divider N+4 reflection logs (for example it would be 6-renection 
logs) 
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Figure 2 

Portfolio Dividers For Example Competency Matrix in Appendix B 



4, Punch holes in your competency matrix and insert it behind the first divider. 
Your Portfolio is now prepared and ready to use. It has N+S dividers (counting the divider for the 
competency matrix), the competency matrix, but no work products as yet. The next section explains 
hov/ to place your work products into your Portfolio. Figure 2 shows what the dividers would look 
like for the sample Portfolio for the class using matrix in Appendix B. 

Filling the Portfolio 

You now have an empty Portfolio ready for your work products. How this might be approached for 
the class using the matrix in Appendix B. Assume you have been given an assignment at the end of a 
Section that requires you to differentiate five trigonometric functions, three of which required implicit 
differentiation, and that you completed the assignment, presenting your work according to the 
expectations of the course instructors^. The question is: Where does this work go? 

This question may be answered in one of two ways. First, your instructor may have explicitly told 
you what Learning Outcome this assignment is related to (e.g., this assignment involves the Learning 
Outcome of differentiation), in which case you now know in which section of the portfolio (i.e., 
differentiation) to place the work product. On the other hand, if the instructor has not provided this 
information, you will need to determine the Learning Outcome. To do this you must find the Uarning 
Outcome that covers the w )rk and put the work in that section of the Portfolio . In our example, the 
Competency Matrix shows that there are only two possible Learning Outcomes: differentiation or team 
meetings. There is little question that this work product is associated with differentiation; thus, the 
work belongs in Section 1 of the Portfolio. 



l^his means you have prepared your work according to the expectations for homework discussed in Part II of this guide 
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This example was relatively easy but what do you do when it is not so clear-cut? The Competency 
Matrix can help you decide if a particular Learmng Outcome is related to the work product In your 
Competency Matrix the Learning Outcomes are further defined as rows within the Learning Outcome, 
In our example, the course learning outcome differentiation is meant to include five things (exponential 
functions, hyperbolic functions, imphcit (differentiation), polynomials, and trigonometric functions). 
For our example, this break down shows even more clearly that the work just completed truly is 
associated with the first outcome and belongs in Section 1 of the Portfolio. 

Before you punch holes in the work and insert it into the Portfolio you need to add some page 
numbers. If this is the first work done in the section, then number the pages 1.1, 1.2, 1.9 
(assuming the assignment was nine pages long). If there is already work in the section find the last 
page number and start with the next number for this new work. Thus, if the last page number were 
1.45, you would number this latest work as 1.46, 1.47,..., 1.54. Notice that the page number 
includes both the section number (1) as well as the page number (45, etc.). Once the page numbers 
have been affixed to the work, punch three holes in the work and add it to yo\ir Portfolio. 

Problems in Determining the Portfolio Section 

The process just discussed is reasonably straight forward; however, questions do arise. Below are 
two common questions with some suggested solutions. 

1 . / cannot find a Learning Outcome that is associated with the work I have just done 

You need to look again, more closely, at the Competency Categories (the rows in the matrix) 
to make sure there are no rows addressing the work product. If you cannot find any row that 
you feel is related to your work product, you need to discuss this with the course instructor. 
You either do not understand what some of the rows actually mean or you have done work 
beyond what was expected by the course instructor. 

2 . The work I have done seems to be associated with more than one section 

This is not an uncommon situation. You could duphcate the work product and put copies of 
the work in each appropriate section, but this is not really needed or desirable. You should 
select one section and put the work product in that section. There is no fundamentally correct 
way of selecting the section but you might consider putting the work product in the section 
that has the most matrix rows addressed by the work product. 

Now you have a Portfolio containing appropriately prepared and presented work products; however, 
but you still have not created the index, the last step and the subject of the next section. 

Building the Index 

This is the most difficult part of the process. You need to enter on your Competency Matrix the page 
numbers of the work products you have just placed in the ponfolio. Where in the Matrix do these page 
numbers go? 

This question is already partially answered. To determine where in the Portfolio to put your work 
products you had to decide on a Learning Outcome and probably the rows within the Outcome 
associated with the work. Thus, for our example, we know the matrix rows of interest are rows 1.3, 
and 1,5 (imphcit, trigonometric functions). The question now is where along the row (i,e,, in which 
matrix box) do you enter the page numbers? 

This question may be answered explicitly by the course instructor (e,g,. This work, when done 
correctly, shows Comprehension for implicit and trigonometric fiinctions) in which case you would 
enter page number 1,46 (the starting page number for the new work) into the matrix boxes located at 
the intersection of rows 1,3, and 1,5 and Comprehension column. 

When the instructor does not tell you the matrix location for the work you must determine this for 
yourself, which is the start of the self assessment process. To determine the matrix location you must 
understand the meaning of the nine columns under the Affective and Cognitive tities and the 
distinctions between them. As pointed out in Part I these columns represent increasing levels of 
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understanding (cognitive columns) and interest or attitude (affective columns). Some of the 
characteristics or traits of work done at each of these levels is given in Appendix A to help you make 
this decision. 

Once you begin to understand these definitions, the question of which box to put the page numbers in 
can be answered: You put the page numbers in the box under the highest level of understanding or 
interest justified by the work as presented in the Portfolio^ ^ In our example, after reading the 
discussions in Appendix A, it seems reasonable that the work shows Responding and Comprehension 
(both of these levels apply to working homework type problems) and you should put page number 
1.46 into the boxes at the intersection of rows 1.3, and 1.5 and the Responding and Comprehension 
columns. 

Problems in Determining the Appropriate Matrix Column 

This process takes some practice and you will need guidance from your course instructor. Here are 
some common questions with some possible solutions. 

1 . When I look at my work and at the definitions of the levels and degrees in Appendix A, I see some 
matching at a particular level but the match does not seem to be complete 

This will occur frequently. The material in the Appendix is rather broad and covers a number 
of different ways (activities) that show you are operating at the level of interest. It is not 
expected that your work will match all aspects of these definitions. 

2 . It seems to me that I have matches in several different levels or degrees, which one should I claim? 

This will occur rather frequently. The levels and degrees are based on taxonomies^^ which 
means the higher levels and degrees grow out of or upon the lower levels and degrees. You 
will find that the difference between the upper and lower levels and degrees is generally the 
environment within which you did the work^^ or how you present your work rather than the 
work product itself. When two levels are possible you should always choose the lower level 
if it has not already been documented (i.e., you have not already placed a page number in the 
lower box) and the higher level if the lower level has already been documented. 

3 . / think that I have been working at a high level but when I look at the work in the Portfolio I do not 
think it really shows that I am working at the high level 

This will happen and must be addressed. First, you can discuss this with the course 
instructor, she may be willing to accept an oral demonstration and let you put the page 
number of the work product that does not really show the high level in the high level box. 
An alternative is to add to your presentation of work so that the presented work does show 
that you were working at the high level. 



^^The as presented is important because for assessment il is only ihe work present in the Portfolio that can be used to 
establish the appropriate level. 

^^Thc systematic distinguishing, ordering and naming of type groups wiihin a subject field, Webster's Third New 
International Dictionary 

^^For example, how much were you told about how to do the work and how much did you have to gin up yourself. 
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What Now? 

When you created your portfolio structure you were asked to create several sections that have not yet 
been addressed. The section on Class notes is self explanatory but what about the others (Work Logs, 
Run Charts, Reflection Logs)? Should you be putting material into these sections? These are 
discussed in the next Part of this guide. If your course instructor asks you to keep Work Logs or Run 
Charts, you will need to read about them in Part IV. You will not initially be asked to create Reflection 
Logs and so you do not need to worry about them at this time. 

After you have experienced the process of preparing work products and entering those products into 
your Portfolio a few times, you will then find it worthwhile to read Part IV to get a more in depth 
coverage of this evaluation process. 
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The concept of an educational state was introduced in Pan I; an introduction to a method that could be 
used to document (verify, show) that you have reached some educational state was presented in Part 
ni. Now it is time to better understand the assessment process so you can master the self evaluation 
process. 

Two important questions need to be answered with regard to assessment: 

How do I know when I have reached some educational state? and 

What do I have to do to show myself or someone else that I have indeed reached this 
educational state? 

When you become an expert at self evaluation you can, and will, fairly and correctiy evaluate your 
actual educational state with little more than internal (i.e., in your brain) evidence. But this type of 
psychic evidence will not be acceptable for classes or other situations where you must show others that 
you have reached a claimed educational state. In these situations you must supply documentation 
(information presented in some appropriate form) that demonstrates (shows) to the satisfaction of 
the reader that you have actually reached the educational states you are claiming. Even in those 
situations where you are doing only self evaluation, the preparation and presentation of this 
documentation generally leads to u fuller understanding of the material (see General Presentation Traits 
in Pan II). 

Before discussing the evaluation process it is necessary to understand the various documentation 
instruments (methods) that are used in the evaluation process; therefore, these instruments are 
discussed first, followed by a discussion of the process. 

Documentation Instruments 

The documentation process uses several different instruments (methods) for recording and storing 
your work. You must understand what each of these instruments is used for and how to use them 
before you can effectively prod»xe them. 

A Portfolio (or Design Notebook) 

A portfolio is an organized collection of your work, technical and non technical, prepared for a 
particular class. It contains, in some logical sequence, all your out of class work, quizzes, tests, 
reports, projects, reflection and work logs (see below), class notes (optional); that is, everything done 
during the semester that relates to the class. Your portfolio becomes a collection of worked examples, 
examples you can refer to in later classes (or when out of school) when you need to review a topic 
learned in an earlier class. 

Physically, portfolios are three hole, loose leaf binders, although accordion files can also be used. 
Loose leaf binders (as opposed to other binders ) are recommended since it is possible to remove 
and/or insert material, a necessary capability. The number and order of the sections in the portfolio is 
not mandatory; however, the following suggested organization should help if you are not sure how to 
start. Note, you may not understand what some of following sections are until you read on but once 
you do understand, you will then know where to put them. 

Table 1 on the next page shows a suggested organization for your portfolio. As you can see the 
number of sections is not fixed, rather they are a function of the Competency Matrix associated with 
the course. Once you understand your Competency Matrix you should be able to find a location (i.e., 
an appropriate section) in the portfolio for all of your work products. 
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The work within a section should also be organized so that you can easily find it. To this end you will 
need to number all the pages of your work. Your page numbering must be unique (i.e., no two pages 
can have identical page numbers). This is probably most easily accomplished by using a numbering 
scheme that includes the section and location with in section (e.g., 1.2 is page two within section 1). 

A design notebook is a special type of portfolio, one containing all the technical work related to a 
design project. Since design projects are often done by teams, the design notebook will include work 
from a variety of people. The sections in a design notebook will not be Learning Outcomes but rather 
separate technical tasks that are undertaken during the design process. Thus, for example, you could 
have sections related to Circuit Analysis, Structural Analysis, Propulsion, etc. 

All the technical work in the portfolio/notebook must to the adhere to the expectations for presentation 
of technical work presented in Part 11. 

Competency Matrix 

In any class the instructor has a set of knowledge and skills (e.g.. Engineering Design Process, 
Second Order Differential Equations, First Law of Thermodynamics, Teaming, etc.) that she wants to 
have the class learn. This set of knowledge and skills are known as the learning outcomes for the 
class. Learning outcomes are generally rather abstract and must be characterized by (i.e., defined in 
terms of) a number of more specific topics called competency categories. Depending on how specific 
the competency categories are, it may be possible (desirable) to further divide these competency 
categories. 

However, it is not enough to define the learning outcomes and competency categories; the instructor 
must also decide, for each of these items, what cognitive and affective behavior levels the students 
should achieve. This collection of concepts to be learned and the levels at which they are to be learned 
can be organized and presented in a Competency Matrix. 

The general design of the matrix is quite simple. Along the left side of the matrix are the general 
course Learning Outcomes (sections in ponfolio) along with each Outcomes' more specific 
Competency Categories (and Sub-categories if they exist). Along the top of the matrix are the various 
affective and cognitive behavior levels. Each cell in the matrix represents the intersection of a 
particular competency category or learning outcome and a particular cognitive level or affective degree. 
Part of a Competency Matrix for a Calculus Class is shown in Appendix B. 

Note that the Competency Matrix includes black dots, 
gray areas, and white areas. The black dots show the 
cognitive and affective performance you are assumed 
to have reached when you start the class. The gray 
areas are the cognitive and affective performances that 
you are expected to achieve during the course of the 
semester. The white areas represent the cognitive and 
affective performance you may achieve, but are not 
those explicitiy expected to be achieved in th 
particular class. 

At the start of a semester your matrix is empty. 
During the semester you will make entries in each of 
the gray (and perhaps white) boxes in the matrix. 
These entries are pointers to work products in your 
portfolio that you feel support your claim of being at 
the specific educational state. The Documentation 
Process Section (see below) describes the information 
that should be noted in these boxes. 

A.s a vehicle for documenting your educational state 
yuu will find that the matrix serves two purposes. 



Section 
Number 


Content 


0 
1 


Course Competency Matrix 
First Learning Outcome in 
Competency Matrix 


N 
N-f 1 


Last of N Learning Outcome in 
Competency Matrix 

Class Notes 


N 4- 2 


Work Logs 


N-f 3 


Run Charts 


N-f 4 


Reflection Logs 


Table 1 

Suggest Portfolio Organization 
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1 . It indicates the cognitive and/or affective performance you have achieved in each of the course's 
competency categories, and 

2. it indicates the location of the technical work which supports the claimed educational state. 
Work Logs 

Work logs, as the name implies, document the time spent preparing the work products you have 
prepared during the semester (i.e., the time spent on various class related tasks) . A sample work log 
template is shown in Appendix B. The log contains factual information related to your work. The log 
tells when the work was done, how much time was spent on the work, where the work is located in 
the Portfolio, and finally a code that clarifies the type of work performed. 

Run Charts 

It is sometimes necessary to monitor, over time, a quantity whose value changes with time (e.g., the 
outdoor temperature). In such situations it is often useful to present not only the current value of the 
quantity (e.g., temperature right now) but also the running, time averaged value of the quantity (e.g., 
average temperature over the last six hours). The name Run Chart comes from this type of 
presentation where you present the running history of the quantity's value. The time changing values 
can be periodically entered into tables. However, trends are hard to see in tables; therefore, this data 
is generally also shown on a graph, known as a Run Chart. A run chan is just a graph or plot that has 
time as the horizontal axis, and the value of the time varying quantity on the vertical axis. The chart or 
plot must show the instantaneous values as well as the time averaged value. Generally the time 
averaged value will be shown as a curve superimposed on the instantaneous values. 

A sample Class Attendance Run Chart that might be used as a general demonstration of Responding in 
a class, is shown in Appendix B. The chart shows time along the horizontal axis (i.e., class number) 
and the value of attendance along the vertical axes (yes = 1, no = 0). The data for each class is shown 
by the height of the bar while the running average is shown by the line. In the example shown the 
person missed class on the 3rd, 5th, and 1 1th day (bar has zero height). For the third class the 
running average dropped from 1 (perfect attendance) to 0.67 (attendance for two of three classes). The 
running average drops for each class not attended and slowly rises when classes are attended. You can 
see that the running average after twenty classes is about .86. 

Reflection Logs 

The last of the documentation tools are the Reflection Logs. These are the most difficult documents to 
produce and should only be attempted after you feel very comfortable with the evaluation process. In 
fact, the ability to create these logs is really di final test to demonstrate that you really are able to do 
evaluation. Why are these logs so difficult to create? 

The work contained in your portfolio displays activities you have undertaken during the semester. 
Some of this material shows Comprehension activities, some shows Evaluation performance, some 
shows Valuing behavior, etc. Traditionally, it has been the course instructor's task to sort through the 
work and decide what educational state was shown by the work. Determining the educational state 
was done by (mentally) comparing the work with a set of Standards (Exemplars ) to decide which 
Standard (Exemplar) the work most closely matched. 

In the self evaluation mode, this task of matching work to performance Exemplars is done by you. 
You must look at the work, as presented in the portfolio, and select the most appropriate educational 
state Exemplar. Once you have completed this mental comparison and selection (i.e., this reflection on 
the presented work) you are ready to write a Reflection Log (see Appendix B for a template) which is a 
written explanation of why you selected a particular Exemplar. Such logs will contain traits from the 
selected Exemplar that you observed in the presented work. The logs often also show why the work is 
not a member of other possible Exemplars. These logs are meant to be informative, describing how 



^^Exemplar - that which serves as a pattern, especially an idc^l pattern 
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you determined the educational state. When well written, they will be persuasive, convincing other 
individuals that you do understand the evaluation process for the educational state selected. 

The log need not look like the template shown in Appendix B, but it should contain the same 
information. The log must have an identifying entry number and it must be clear which competency 
category (i.e., row(s) in the Competency Matrix) and Level of Learning and/or Degree of 
Internalization (i.e., columns in the Competency Matrix) ai'e being addressed. Ad^tionally, it must be 
clear where in the Portfolio the technical work being discussed is located. Finally, the log must 
contain the paragraph(s) of reflection. You may find that the reflections require more space than is 
given on the form, in which case you can append the necessary additional pages to the log entry. 

The Self Evaluation Process 

Now that you are fanoihar with the various documentation instruments it is possible to discuss the 
process that will let you complete your Competency Matrix (i.e., let you fill in each gray box). This 
section describes a general, self evaluation process, along with two slightly different implementations 
of the process. 

The general self evaluation process consists of the following six steps: 

1 . generate and present evidence of activities or abilities, 

2 . compare your evidence to a set of Abilities or Activities Exemplars defined using a 
predefined hierarchy of learning, 

3 . select the Exemplar that best fits your evidence, 

4 . explain how the selected Exemplar defines your current educational state, 

5 . explain your Exemplar selection process, and 

6. fill in the gray boxes in the Competency Matrix. 

Details about step 1, steps 2, 3, and 4, taken together, and steps 5 and 6 follow. 

Generating and Presenting Evidence (step 1) 

Before steps 2, 3, and 4 of this process can be completed there must be activity and memorialization of 
the activity. Tlius, the first step in documenting your educational state is the generation, collection, 
and presentation of evidence that can be used to show that you have indeed reached a particular 
educational state. The specific work product you decide to produce (i.e., the problems you solve, the 
way you solve them, and the way you present them) is influenced by the motivation you have for 
doing the work. Several different motivations are possible. First, the work may be motivated strictly 
by your desire to have examples showing that you are at some educational state (e.g., doing out of 
class assignments assigned by the class instructor). Second, the work may be generated strictly 
because you are solving a self-generated, technical problem that you have an interest (need) in solving 
(e.g., working on a design project). Third, the motivation may be a combination of these two (i.e., 
working to solve self-generated problems and, incidentally, also using the work to show achievement 
of some educational state). 

For courses that are not project intensive, most of the work done in the class is done for the first 
reason, generation of examples showing that you have learned}^ the material. This sounds reasonable; 
however, few assignments include a notation explaining which gray boxes in a competency matrix 
should be marked to show that the assignment illustrates mastery of a competency category Generally 
an assignment will indicate mastery of several competency categories; with a little imagination, most 
assignments can be used for a wide variety of competency categories and several different levels or 
degrees. You can do an assignment to show Knowledge or Analysis depending on how you choose to 
perform and present the work. 



^^achieved some cognitive and/or affective behavior level 
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Learning to map assignment to matrix and matrix to assignment (i.e., learning to recognize the learning 
potential in an assignment ) is not necessarily intuitive and takes some skill and thought When you 
are first learning how to do self evaluation, the course instructor will discuss with you how her 
assignments are related to the matrix and what gray boxes you could reasonably use. As you gain 
familiarity and skill at doing self assessment you will find the course instructor will tell you less and 
less, letting you do more of the determination of the assignment's potential. You should eventually 
reach a point where you actually mold the assignment to make it fit your immediate matrix needs. 

For project intensive courses the work performed is primarily of the second kind, i.e., self generated 
problems, presented according to expected standards, and located in a Design Notebook. However, 
since the work is pan of a course wiUi specified educational states (gray boxes), some of the technical 
project work will also be proof of achieving these specified states. You will find that, for some of the 
educational states you are striving to demonstrate, the standard (Vanilla') notebook presentation of the 
technical work is not sufficient; the 'vanilla' presentation fails to show the educational state at which 
you were actually operating. It is relatively easy to present work that actually requires critical thinking 
(Analysis, Synthesis, Evaluation Levels of Learning) in a form that looks like Comprehension. In 
such cases you will need to augment the standard ('vanilla') presentation so the work more clearly 
reflects the educational state at which you were really working. 

Exemplar Selection (steps 2, 3, and 4) 

The Exemplars you are to use in steps 2, 3, and 4 are the various Levels of Learning and Degrees of 
Internalization defined by the two educational objectives taxonomies. You are to use the write-ups 
found in Appendix A as the definitions of these behavior states. The task of selecting the appropriate 
Exemplar consists of reading the various definitions and descriptions of activities associated with each 
Exemplar and comparing these activities with the activities that have been presented in the previous 
generation step. 

For example, suppose you were in a Calculus class, working assigned, single concept problems, 
located at the end of a section conceming instantaneous velocity; what educational state does this type 
of activity suggest? Consider the cognitive domain first. Appendix A shows that for Knowledge, you 
would be expected to: define terms, read material, take simple recall quizzes. None of these activities 
seem to match what you have been doing. Reviewing the next Exemplar, Comprehension, you see 
descriptions such as: work assignments in which the appropriate approach is evident, describe the 
results of working a problem, restate an idea in your own words. You also see that a typical work 
product is well presented homework. This seems to be a better description of your work; at least there 
are some matches (e.g., worked problem, discussed results). However, before selecting this 
Exemplar you should consider the next higher one. Application. Here you see that the activities are 
similar to those of Comprehension with the major difference being the source of the solution method 
used is not immediately evident. You must now decide whether the method you used in solving the 
problem was given to you or whether you remembered the method from some other source or time. 
Problems at the end of chapters and sections are almost always of llie first type; thus, it seems clear 
that this work is evidence of working at the Comprehension level for competency categories associated 
with instantaneous velocity. 

What about the affective domain? The process is identical. Consider the lowest degree. Receiving. 
Appendix A illustrates that at Receiving you: listen, read, make notes of topics or items contained in 
the lecture, reading or classroom activity. This does not sound like what you have been doing; it is too 
passive. You have been doing. Reviewing the next degree Responding, you see: calculating, 
answering questions in class, working instructor assigned problems. This seems promising; again 
there is at least a partial match between what you did and the list of possible activities (i.e., doing 
assigned work). Reviewing Valuing, you see that at this degree you: believe the ideas of instantaneous 
velocity helps solve problems^ can justify this belief, and are concerned when people express 
skepticism about instantaneous velocity. This Exemplar is possible and you will have to determine 
whether you are just responding or have developed a value for the material. A Calculus Course is 
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likely to be your introduction to the concept and you probably will not have developed any of the 
characteristics associated with Value (i.e., you are at the Responding educational state). 

As another example, suppose you have created a report that explains the process you used in designing 
a bridge for a team design project. What cognitive level for failure analysis might this report and 
accompanying technical work support? A quick review of the lower cognitive levels shows partial 
matches for Q)mprehension and Application with Application seeming to be the best choice since you 
were not told exactiy how to design the bridge and so had to recognize the utility of the solution 
methods you used. Reviewing the Analysis Exemplar, you see: able to break into parts, able to 
explain why the process works, able to explain causal relationships among tiie parts. You also see that 
the typical work product includes a report and in reading your report you recognize traits that match 
some of these tilings (e.g., breaking problem up into logical parts for studying different failure 
mechanisms). Review the next Exemplar, Synthesis, where you see: put together parts, create a 
report or a process. Again, the material presented in the report shows at least a partial match, you have 
created the analysis process you used and, further, have explained this process in a report that you also 
created. So Syntiiesis is shown; is Evaluation justified? Reviewing Evaluation you see activities such 
as: compare, choose, evaluate, criteria, etc. If you considered several, different, analysis methods 
and selected one of die methods based on a rational set of criteria, tiien you are operating at this top 
level; if you did not compare several methods but rather created a single method, then it would seem 
you were at the Synthesis and not Evaluation level for failure analysis. 

Note: the decision about the higher levels of learning was based heavily on what was and was not in 
the report that tied the technical work together, and not so much on the presentation of the technical 
work. It is entirely possible that if you were to only present the technical work, the only conclusion 
you could reach about the level of learning shown would be Comprehension. 

It is difficult to determine the appropriate affective degree; you must read the descriptions and decide 
which is appropriate. It will be either Responding or Valuing; however, which it is depends on you. 
It is possible tiiat, if this project is a Senior Project, you have used these failure analysis methods 
enough that you do believe they are useful, in which case you are at the Valuing degree. If you do not 
see them as useful, then you are probably still at the Responding degree. 

You can tell from these two, rather brief examples, how you might go about selecting an appropriate 
Exemplar. These examples also show that the matches between work presented and the various 
Exemplars will not necessarily be exact. You will have to make some decisions about when there are 
enough matches to justify the claim. 

Explicit Justification of Exemplar Selection (step 5) 

You must go through the previous comparison/selection process for every gray box in your 
Competency Matrix and every piece of work you present, mentally comparing what you did and have 
presented with die Exemplars. Initially, because self evaluation is a new to you and because you are 
not familiar with the Exemplars, this process will take time and you may not do it correctiy. In most 
cases, during these learning stages of the process, your course instructor will tell you what Exemplar 
is possible when you work a problem. In this case your task becomes one of making sure your 
presentation supports the given Exemplar. As you learn how to generate the appropriate type of 
presentation you will be learning how to do the inverse problem, e.g., recognition of an Exemplar 
from a presentation of work. It is the inverse problem that was discussed in the previous section and 
is what you will need to learn how to do if you are to master self evaluation. 

At some point in the class, after the instructor feels that you have indeed mastered the presentation of 
work for some Exemplar, the instructor may ask you to demonstrate that you truly understand the 
Exemplar, by having you explain why some set of work in your Portfolio shows mastery of the 
competency categories at the Exemplar of interest {t.g., persuade me that you understand what is 
needed to be at the Analysis level of learning). The way you will do this is to create a Reflection Log 
for the work in question. 
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As discussed earlier, a Reflection Log discusses the process you went through in deciding which 
Exemplar most closely matches the presented work. A Reflection Log is not a discussion of the work 
that is done as an expected pan of the presentation of the work (see Part II). Rather, a Reflection Log 
focuses on the general characteristics or traits of the work, characteristics or traits that allow you to 
match the work with an Exemplar. For example, if you were writing a log for the previous bridge 
design report work and were making a case that the presented work supported a claim of Synthesis 
level of learning you would need to look for: 

1 . the existence of well presented technical work 

(see Typical Work Products and Process Verbs for Synthesis) 

2. the existence of a report or work that tied all the technical work together 
(see Typical Work Products for Synthesis) 

3. discussion that was related to the creation of something 
(see Typical Questions and Process Verbs for Synthesis). 

In the Reflection Log you would want to point out where in the presented work you could find 
examples of these three things. Thus, for example, you could point out the existence of the report as 
one of the work products supporting your claim of Synthesis. You would need to point out sections 
of the repon where the discussion focused on creation (addresses issue 3). You would probably also 
want to point out the absence of work products associated with Evaluation. 

How Many Reflection Logs 

For this class you will only be writing Reflection Logs when the instructor so requests and this will 
usually happen after you have demonstrated in your Portfolio, as evaluated by the instructor and >our 
peers, that you can consistently prepare and present work at some level of learning. In other words, 
you will be asked to write these Refection Logs only after you have demonstrated a Comprehension 
Level of Learning concerning the Exemplar of interest. You will write at least one log for each level of 
learning (Exemplar). If the first log fails to show an adequate grasp of the issues associated with the 
Exemplar, (i.e., fails to show you are at Comprehension Level of Learning for the Exemplar) you will 
write another log, for a different body of work in your Portfolio. 

Filling In The Competency Matrix Gray Boxes (step 6) 

This is the last, and by far the easiest, step in the entire process. The filling in of the gray boxes 
should occur at a relatively uniform rate during the semester. Entries in these gray boxes depends on 
whether they are affective or cognitive boxes. 

Cognitive Gray Boxes 

After you have determined the appropriate Exemplar (LoL for the work), you simply enter the portfolio 
(Design Notebook) page vmmbers that contain the evidence supporting your claim. This would be the 
first page of the work product or report. It could also be the page number for a figure or chart within a 
body of work. 

Affective Gray Boxes 

Collecting demonstration material for the gray affective boxes is a bit more difficult, especially for 
Valuing where the Exemplar uses terms like believe and feel, things not easy to capture in your 
presentation of work. This is the first year in which the affective domain has been explicitly used in 
the Competency Matrix and I hope the class and I will learn together about how to document these 
affective boxes. However, until another methods as been determined the gray affective boxes are to be 
handled as follows. 

1. Receiving 

It seems very unlikely that you can operate at the cognitive Knowledge level without 
operating at the Receiving degree. Thus, by inference, you can claim proof of Receiving any 
time you justify Knowledge. Put the same Portfolio page number entry in the Receiving gray 
box as you put in the Knowledge gray box. 
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2. Responding 

It again seems very unlikely that you can operate at the cognitive Comprehension level 
without operating at the ResponcUng degree. Thus, by inference, you can claim proof of 
Responding any time you justify Comprehension . Put the same Portfolio page number entry 
in the Receiving gray "box as you put in the Knowledge gray box. 

3, Valuing 

These boxes are a bit more difficult. Presentations of technical work, even high quality 
presentations, do not contain the sort of information that would allow a person to deduce 
whether or not you valued the process or methods being used. If you feel you have reached 
this degree of internalization then you will need to prepare a Reflection Log that explains why 
you have reached this degree. This Valuing Reflection Log will contain information not 
found in the presentation of the work, but germane to establishing your claim of Valuing. 

It may be that you do not feel that you have reached the Valuing degree by the end of the 
semester. If this happens, prepare a Reflection Log that discusses why you have not reached 
Valuing and refer to that log in the Competency Matrix. 

For these Valuing Reflection Logs, you should enter the Reflection Log Page number into the 
gray box (you can also enter the Reflection Log item number in the gray box if there is room) 

Run Chart Competency Categories 

You may find some of the competency categories in your Competency Matrix have been marked as 
run chart categories. This designation is given to those categories that cannot be documented with a 
single effort but require a semester long accumulation of efforts to show achievement of the category. 
For such categories you will be maintaining a run chart that shows your semester long commitment 
and, at the end of the semester, you will place that run chart's page number in the competency matrix 
box. 

When Can You Claim Level or Degree Mastery? 

If for some competency category you show you can operate at an educational state once, does that 
mean that in this class you would always operate at that state? That is, if you were to work a second 
problem using the same competency category would the second effort always be at the same 
educational state as the first effort? If the answer is maybe or maybe not, then how many times must 
you perform at some educational state before you can safely claim retention of the level (i.e., claim 
mastery ^^of the educational state). 

This discussion of mastery is limited to the time period you are in this class and does not include what 
happens to your educational states once you leave the class. There is no question that, if you do not 
subsequently use the competency category items in future classes or work related activities, the 
educational states you achieve in this class will decay. The curriculum, of which this class is a part, 
has been developed with an awareness of this time decaying nature of educational states and has been 
constructed to give you many opportunities, in a series of classes, to use the competency categories 
that need to have continual growth (or at least no decay) to support educational states. The expected 
educational states shown in the Competency Matrix for this class are consistent with either the 
educational states needed in further classes or the desired, BS graduate, life long levels of 
performance. 



^^Noie this material is using mastery of a competency category at a level or degree and not mastery of the category , 
which generally implies being able to work, unconsciously at the highest cognitive level, i.e., being an expert of the 
category. Mastery , as used in this section, also does not have any performance speed atuiched lo the meaning. Mastery 
does not mean that you have instant recall or the ability to instantly solve the problem at the educational state claimed. 
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Activities at Various Levels of Learning 



KNOWLEDGE (INFORMATION) 

How do I know I have reached this level? 

I recall information. I bring to mind the appropriate material at the appropriate time. I have been 
exposed to the information and can respond to questions, tasks, etc. 

What do I do at this level? 

I read material, listen to lectures, watch videos, take notes and I am able to pass a test of knowledge on 
the subject area. I learn the vocabulary of the competency area, i.e., the terminology. I leam the 
conventions used. 

How will the teacher know I am at this level? 

The teacher will provide opportunities (either orally or in written tests), regardless of complexity, that 
can be answered through simple recall of previously learned material. 

What does the teacher do at this level? 

The teacher directs, tells, shows, identifies, examines the information necessary at this level. 

What are typical ways I can demonstrate my knowledge? 

1 . Define technical terms by giving their attributes, properties, or relations. 

2 . Recall the major facts about a panicular subject 

3. List the characteristic ways of treating and presenting ideas (i.e., list conventions associated with 
the subject) 

4. Name the classes, sets, divisions, and arrangements that are regarded as fundamental for a given 
subject field or problem. 

5 . List the criteria used to judge facts, principles, ideas. 

6. Describe the method(s) of inquiry or techniques and procedures used in a particular field of study. 

7 . List the relevant principles and generalizations 

8. Fill in the blank 

What are typical work products? 

1 . hand written answers to Knowledge quizzes (True/False, Yes/No, multiple choice, fill in the 
blank) 

2. lists of definitions 

What are descriptive Process verbs? 

define label listen list 

memorize name read recall 

record relate repeat view 
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Activities at Various Levels of Learning 



COMPREHENSION (UNDERSTANDING) 

How do I know I have reached this level? 

I comprehend and understand what is being communicated and make use of the ideas but without 
relating them to other ideas or material. I may not yet understand the fullest meaning. I understand 
what others are discussing concerning this idea. This level requires Knowledge. 

What do I do at this level? 

I successfully work assignments in which the appropriate approach is evident either because of 
material in the problem statement or because of the problem's relative location in the book to the 
appropriate method. I translate information into my own words (translation from one level of 
abstraction to another). I translate symbolic information (e.g., tables, diagrams, graphs, mathematical 
formulas, etc.) into verbal forms, and vice versa, I interpret or summarize communications 
(written/graphical/oral). I determine implications, consequences, corollaries, effects, etc. that are 
extensions of trends or tendencies beyond the given data. 

How will the teacher know I am at this level? 

The teacher will often ask questions or give tests that can be answered by merely restating or 
reorganizing material in a rather literal (clearly stating the facts or primary meaning of the material) 
manner to show that I understand the essential meaning, e.g., give the ideas in your own words. 

What does the teacher do at this level? 

The teacher demonstrates, works problems, listens, questions, compares, contrasts, and examines the 
information and your knowledge of ii. 

What are typical ways I can demonstrate or can show on my own my comprehension and 
understanding. 

1 . Read (Comprehension type) problems, know what is being asked for, and successfully work the 
problems 

2. Clearly chronicle the process used in working the problem 

3 . Clearly describe the results of working a problem 

4. Draw conclusions (interpret trends) from the results of solving a problem 

5 . Compare/contrast two different problems (i.e., what things are the same?/what things are 
different?) 

6. Restate an idea, theory, principle in your own words. 
Wha t are typical work products? 

1 . hand written answers to Comprehension quizzes and exams (problems, multiple choice) 

2. work associated with homework problems, presented with enough information that it is clear what 
the problem is, why it is being worked, what general method is being used, and what the results of 
the work mean 

What are descriptive process verbs? 
describe explain express identify locate recognize report 

restate review tell work 
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Activities at Various Levels of Learning 



APPLICATION (THINKING) 

How do I know I have reached this level? 

I have the ability to recognize the need to use an idea, method, concept, principle, or theory without 
being told to use it, i.e., I have the ability to use ideas, methods, concepts, principles and theories in 
new situations. I know and comprehend the information and can apply it to a new situation. I also 
have the ability to recognize when a certain task, project, theory or concept is beyond my current 
competency. Application requires having Knowledge and Comprehension. 

What do I do at this level? 

I work problems in which the solution method is not immediately evident or obvious. I take 
knowledge that has been learned at the Knowledge and Comprehension Level of Learning and apply it 
to new situations. I solve problems on my own and make use of other techniques. This requires not 
only knowing and comprehending information, but deep thinking about the usefulness of this 
information and how it can be used to solve new problems that I create or identify. 

How will the teacher know I am at this level? 

I will show the teacher through my work that I am involved in problem solving in new situations with 
minimal identification or prompting of the appropriate rules, principles, or concepts by the teacher. 
The teacher will be able to ask general questions like. How much protection from the sun is enough 
and I will know how to attack the problem. 

What are the typical ways I can demonstrate or show, on mv own, my application of knowledge 
and understanding? 

1 . Solve problems that require recognition of the appropriate concepts, theories, solution techniques, 
etc. 

2. Apply the laws of mathematics (chemistry, physics, engineering) to practical situations 

3 . Work project type problems 

What are typical work products? 

1 . work that looks very much like that produced for Comprehension plus some additional information 
that shows you recognized the need to use the solution methods utilized 

What are descriptive process verbs? 

apply demonstrate employ illustrate 



mterpret 



operate 



practice 



recognize 
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Activities at Various Levels of Learning 



ANALYSIS (THINKING) 

How do I know I have reached this level? 

I can explain why. I can examine, methodically, ideas, concepts, writing etc. and separate into parts 
or basic principles. I have the ability to break down information into component parts in order to make 
organization of the whole clear. Work at this level requires having Knowledge and Comprehension 
Levels of Learning (Application is not required). 

What do I do at this level? 

I analyze results by breaking concepts, ideas, theories, etc. apan. I can explain the logical 
interconnections of the parts and can develop detailed cause and effect chains. 

What does the teacher do at this level? 

The teacher probes, guides, observes, and acts as a resource. 

What are typical questions I can pose for myself to answer that will demonstrate or show mv 
Analysis Level of Learning? 

1 . Why did this (result) happen? 

2 . What reason does she give for her conclusions? 

3 . Does the evidence given support the hypothesis, the conclusion? 

4. Are the conclusions supported by facts, opinions, or analysis of the results? 

5 . What are the causal relationships between the results for the whole and the parts? 

6. What are the unstated assumptions? 
What are typical work products? 

1 . hand written or word processed answers to Analysis exams (problems, multiple choice, essays) 

2. work that looks very much like that produced for Comprehension but it has some additional 
discussion of the work. The type, amount, and depth of discussion (see items above) is what 
distinguishes the work from Comprehension 

What are descriptive process verbs? 

break apart break down examine explain 
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Activities at Various Levels of Learning 



SYNTHESIS (THINKING) 

How do I know I have reached this level? 

I have the ability to put together parts and elements into a unified organization or whole that requires 
original, creative thinking. I recognize new problems and develop new tools to solve them. I create 
my own plans, models, hypotheses for finding solutions to problems. This Level of Learning requires 
Knowledge, Comprehension, Application and Analysis Levels of Learning. 

What do I do at this level? 

I put ideas together to create something. This could be a physical object, a process, a design method, a 
communication, or even a set of abstract relations (i.e., mathematical models). I produce reports 
(written/oral) that create a desired effect (e.g., information acquisition, acceptance of a point of view, 
continued support, etc.) in the reader (listener). I generate project plans. I propose designs. I 
formulate hypotheses based on the analysis of pertinent factors. I am able to generalize from a set of 
axioms, principles. 

How will the teacher know I am at this level? 

I show that I can combine ideas into a statement, plan, product, etc., that is new for me; e.g., can I 
develop a program that includes the best parts of each of those ideas? 

What does the teacher do as this level? 

The teacher reflects, extends, analyses, and evaluates. 

What are the typical questions I can answer that will demonstrate or show my synthesis? 

1 . Can I create a project plan? 

2 . Can I develop a model? 

3 . Can I propose a design? 

What are typical work products? 

1 . hand written or word processed answers to Synthesis exams (problems, multiple choice, essays) 

2. work that looks very much like that produced for Comprehension but it has some additional 
discussion of the work. The type, amount, and depth of discussion (see items above) is what 
distinguishes the work from Comprehension 

What are descriptive process verbs? 



arrange 
construct 
manage 
propose 



assemble 
create 

organize 
set up 



collect 
design 
plan 
write 



compose 
formulate 
prepare 
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Activities at Various Levels of Learning 



APPRECIATION/EVALUATION (WISDOM) 

How do I know I have reached this level? 

I have the ability to judge and appreciate the value of ideas, procedures and methods using appropriate 
criteria. To work at this level requires having achieved Knowledge, Comprehension, Application, 
Analysis and Synthesis Levels of Learning. 

What do I do at this level? 

I make value judgments based on certain considerations such as usefulness, effectiveness, and so on. 
Based on information gained through application, analysis, and synthesis I can rationally select a 
process, a method, a model, a design, etc. from among a set of possible processes, methods, models, 
design, etc. I evaluate competing plans of action before actually starting the planed work. I evaluate 
work based on internal standards of consistency, logical accuracy, and the absence of internal flaws 
(e.g., I can certify if design feasibility has been demonstrated in a report). I evaluate work based on 
external standards of efficiency, cost, utility to meet particular ends (e.g., I can. certify that design 
quality has been demonstrated in a report). 

How will the teacher know I am at this level? 

I can demonstrate that I can make a judgment about something using some criteria or standard for 
making the judgment. 

What does the teacher do at this level? 

The teacher clarifies, accepts, harmonizes, aligns and guides. 

What are typical statements and questions I can respond to that will demonstrate or s how my 
appreciation/evaluation? 

1 . I can evaluate an idea in terms of ... 

2. For what reasons do I favor... 

3 . Which policy do I think would result in the greatest good for the greatest number? 

4. Which of tiiese models (i.e., modeling approaches) is best for my current needs? 

5 . How does tiiis report show that the design is feasible? 

6. How does this report show the quality of the design? 
What are typical work products? 

1 . hand written or word processed ansv/ers to Evaluation exams (problems, multiple choice, essays) 

2. work that looks very much like that produced for Comprehension but it has some additional 
discussion of the work. The type, amount, and depth of discussion (see items above) is what 
distinguishes the work from Comprehension 

What are descriptive process verbs? 

appraise choose compare estimate evaluate 

(quality) 

judge predict rate value select 

(quality) 
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Affective Domain 

RECEIVING (ATTENDING) 

The degree of Receiving ranges from simple awareness, to a willingness to receive, and finally to 
controlled or selected attention. You must first become aware of information, concepts, theories, etc. 
before you can decide whether or not you are willing (and able!) to receive this information, etc. The 
next decision is naturally whether or not this information, etc. is important enough to merit your 
controlled or selected attention. 

What do I do at this degree? 

At this degree of intemahzation I am concerned only with my conscious recognition of the existence of 
certain phenomena and stimuli (i.e., I am willing to receive or attend to these phenomena and stimuli). 
I am willing to listen to lectures, read books, or watch videos and am aware^^ of the material being 
presented in the lectures, books, videos^^. I have written notes or sketches related to the presentation. 
How do I know I have reached this degree? 

At the end of a lecture, book or video, I can select from a set of topics the ones that were presented in 
the material. I can list or briefly summarize points from the presentation. 

What are descriptive process verbs? 
concentrate feel (touch) listen 



pay attention read 
remember smell 



recognize 
watch 



17 



I 



ERIC 



18 



Does not imply Comprehension or Knowledge of the material only awareness 

A point of caution: being willing to receive may not actually ensure that you are receiving. You do not come to class 
de novo. Because of previous experiences (formal or informal), you bring a point of view which may facilitate or hinder 
recognition of the teacher presented phenomena. These a priori points of view (also referred to as paradigms or learning 
Styles) can filter your attending causing you to actually n^ receive the intended material, even when you are willing to 
receive the maieriaL 

Modifications and paraphrasing by B. McNeill and L. Bellamy of David Krathwohl et. al's Taxonomy 
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Affective Domain 



RESPONDING 

The degree of Responding ranges from simply agreeing to respond, to a willingness to respond, and 
finally to actually getting some satisfaction from responding to directions, requests, information, 
concepts, theories, etc. 

What do I do at this degree? 

I leam by doing^^. I answer questions in class when called upon. I do the work assigned by the 
instructor for this class. I attend and participate in team meetings organized by the instructor. Outside 
of class, I spontaneously discuss course material with classmates and especially with friends who are 
not classmates. I ask questions in class. I prepare notes on my reading assignments. I reorganize and 
integrate my class notes with my reading assignment notes. I participate in the learning activities 
created by the instructor, both those inside the classroom and those outside of the classroom. I make 
an effon to find the class material interesting and useful. I may use it as the basis of written or oral 

reports in other classes 

How do I know I have reached this degree? 

I feel comfortable with my team (i.e., I know that I am contributing at least what is expected by my 
teammates). I meet the class deadlines. I invest the expected level of effort for the class. I attend class 
regularly and arrive on time. I have work products or deliverables, that I have constructed for this 
class, that are well organized and accessible. 

What are descriptive process verbs? 
calculate discuss draw 
integrate make organize 
play sketch talk 

write 



As paradigms can color what you actually receive, fear can keep you from responding, even when you are willing . Fear 
of failure (as judged by an insunctor or peer), fear of looking silly, fear of ridicule, fear of missing the point, fear of 
punishment, fear of a poor grade, fear is a powerful, extrinsic DEmotivalcr which you will have to address before you 
reach the Responding degree of internalization. 

^^Responding is still a rather low level of commitment and it would be incorrect to say that you value the material or 
that you actively displayed an attitude toward the material. 
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Affective Domain 



VALUING 

The degree of Valuing ranges from simply accepting the values presented by the instructor, to 
preferring these values, and finally to actually making a commitment to these values. The values of 
interest here are the directions, requests, information, concepts, theories, etc. specified in the written 
course material or orally by the instructor during the course. 

What do I do at this degree? 

I frequently use the material (i.e., skills, methods, knowledge, etc.) that I have learned to solve 
problems. I may try to teach the material to people who are not familiar with the material or people 
who are having difficulty understanding the material. I use the material in activities unrelated to this 
class. I form or take part in study groups to further understand the material. I may challenge people 
who are skeptical about the material in an effort to help them eliminate or at least reduce their 
skepticism. 

How do I know I have reached this level? 

I beheve the material I have learned is useful and helps me solve problems. Also, I can justify or 
explain this belief both to my self and to others. I make a concerted effort to obtain more information 
about the class material (e.g., obtain additional books, watch extra videos, attend other classes or 
seminars). I am concerned when people express their skepticism or doubts about the material and its 
value or usefulness. I am comfortable working in teams or working alone. I am interested in the class 
material and demonstrate curiosity, honesty, integrity and truthfulness in dealing with others and in the 
work products or deliverables I produce for the class. 

Wh ? ^t are descriptive process verbs? 
care convince use 
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Reflection Log Template 



Log Entry Number 
Competency Category(s): 

Level of Learning and/or Degree of Internalization Claimed: 

Location of Supporting Work 

Reflection 



Log Entry Number: 
Competency Category(s): 

Level of Learning and/or Degree of Internalization Claimed: 

Location of Supporting Work 

Reflection 
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Work Log Template 



Log Entry 
Number 


Name of Person 
Doing the Work 




Total Time Spent 
(decimal hours) 


Location of Work in 
Portfolio 


Work 
Code 



























































































































































































































































































































Work Code Definition Work Code Definition 

1. 5. 

2. 6. 

3. 7. 

4. 8. 
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The Environment and Purpose 

❖ Learning Environment 

♦ Active (as opposed to Passive) 

♦ Group or team based 

♦ Workshop facilitators (not Lecturers) 

❖ Purpose and Expected Participant Outcomes 

♦ Learn the Principles of Team Dynannics 

♦ Apply the Principles to Create Effective Teams 

♦ Plan, Lead, and Participate in Effective Meetings 

Getting Started 

❖ Forming Groups 

❖ Tocus on the Facilitator' Signal 

❖ Issue Bin 

❖ Code of Cooperation 

❖ Self Evaluation Matrix 
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Forming Groups 



❖ Groups of 5 or more are appropriate for this 
workshop. 



❖ Take a seat an any table or arrange the 
chairs in the room to accommodate 
5 or more people. 



❖ Make an effort to seat yourself with people 
you do not know 



^Focus on Facilitator' Signal 

The frcilitator needs your attention: 

• Raise your hands to inform your neighbors 

• Finish your sentence 

• Do NOT finish your paragraph 

• Turn toward the facilitator 



S4 



Issue Bin ( a useful tool ! ) 



> Someone will be assigned to be the Issue Bin Collector 
The following issues will be assigned to the Issue Bin: 

• topics that will or may be addressed later 

• questions that can or should be deferred until the end of the 
workshop 

• items that can or should be the subject ' ^ future workshops 

> Paraphrase the issue and record on the board or a piece of paper 
which is always visible 

At the conclusion of the session or workshop, the issues in the 
issue bin are brought out, one at a time, and discussed to see if 
they are still issues. 

Any issues which remain after the discussion must be addressed in 
a future workshop. 
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Code Of Cooperation ^ 



1 . bVERY member is responsible lor the team's progress and success. 

2. Attend all team meetings and be on time. 

3. Come prepared. 

4. Carry out assignments on schedule. 

5. Listen to and show respect for the contrtoutions ol other members; be an active listener. 

6. CONSTRUCTIVELY criticize ideas, not persons. 

7. Resolve conflicts constructively. 

6. Pay attention, avoid disruptive behavior. 

9. Avoid disruptive side conversations. 

10. Only one person speaks at a time. 

11. Everyone participates, no one dominates. 

12. Be succinct, avoid long anecdotes and examples. 

13. No rank In the room. 

14. Respect those not present. 

15. Ask questk>ns when you do not understand. 

16. Attend to your personal comfort needs at ari/ time but minimize team disruption. 

17. HAVE FUN »!! 

18. ? 



adapted from the Boeing Airplane Group team Member Training Manual 



85 




^ J Competency Matrix 



a 



❖ A 'snap shot' of a person's level of learning or degree of 
internalization for a variety of concepts or skills 

❖ An L matrix of concepts, topics or skills to be learned 

versus 

levels ( or stages ) of learning and 
degrees of internalization 

❖ The matrix is filled out by person being evaluated 

❖ It must be constructed by the person responsible for 

(1) establishing the course objectives, and 

(2) designing both the learning experiences and 
assessment instruments required to achieve the 
course objectives. 




Malcing Teams Out of Groups 



a 




Team 
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(^^\ Making Teams Out of Groups : 



Session Structure 



❖ Lecturette on Why Teams are Important (10 minutes) 



❖ Jigsaw on Team Dynamics (125 minutes) 

( presented in two class sessions with breaks ! ) 

❖ Session Process Check (5 minutes) 

❖ Jigsaw Process Check (15 minutes) 



I 
I 
I 
I 
I 
I 

1 Wtiat is a team anyway? 

I 

A team is a small number of people with 

I complementary skills who are committed to a common 
purpose, performance goals, and approach for which 
they hold themselves mutually acountable^ 

I 
1 
I 
I 
I 
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❖ Small Number 

❖ Complementary Skills 

❖ Common Purpose & Performance Goals 

❖ Common Approach 

❖ Mutual Accountability 



1 Jen P Katzer^ac^ & Dou^as K Smith 1993 The wisdcm of t«tms Crettng ihe high-pt>rhm*nca Org4rui4l>on 
C«mb(Kl9» MA H*rviird Bgiir>«» School P(«ss 
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Use of Teams 



In industry /business : 

❖ Management teams (Team Xerox. San Diego Zoo) 

♦ Continuous Quality Improvement teams (CQI) 

♦ Destgn/BulkJ teams (Chrysler H-car, Boeing 777) 

In academe ; 
<► Cooperative learning 

Short-term groups 

Long-term groups 

Base groups 

❖ Project-based cou rses 

Single-disclpiine teams 
Multi-disciplinary teams 
*> Design Courses 

Technical multi-disciplinary teams 
Cross-lunctional teams (marketing, engineering, law. etc.) 




•The task tor us at Boeing b to 
provide a masstvc change 5n thmkjng 
throughout the company - this is a 
cu Rural shitt, and rt isnl easy!' 

PhilCondrt. 
Executive Vice President 
Boeing Commef aai Airplanes 
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Rating of Employee's 
Overall Employment Potential 



Has (J«n>on«t»t^ »cp*nor *»chnicaJ comp«Uoc» in « wid* rinp* of iklls. tiU ifOitYO^ 

oroumiU.'X:** quicWy; mor» ton «p»cted ^hrou^^ C0ft«*l»oly h«rd wortt 4 (J»<ic»oon. 

if\ncv»*v» •o4ulcn* to proWwnt: dwnonttniM » p«Mion for txcMiSng th« ♦^^♦cUboo* of 
ou»tom«f«. communic*to« «o•p•«^•Hy w^J iponttn^ouiiy Vrtti !««• or no pr»p»m»<3o 



Ov*rU R<«ng 

(1 to 100) 



60 

70 
60 
50 



«d«pt* chjripro ciroum»t«^c»s »nd I»»iti« n»w *ils r»»o»9*«ry: 9*ts ftCo*pUbl« r«iutti 
•nd put* r twod f*/* wortt tor • good d*/* p«y * occMioo«lly d#v«lop» «f>ov»»v. sctubont to 
pfoWwns. »how»*mod««»in»»f»»trmM«ng»t«n*»d«o*ou«tomm oommun<c»t«» w**! w*<h 
o*rt«n groups and ot\ timliv topics 



Fsctors ic b« constd«r#<] 
m insking ov*isll r«lng 

AtMudt 
Ablity to LMtn 
Results 

WortcEtttc 

T*<tintctJ Comp«t«nc« 
kviovsloo 

TMmwork 

Communteslon 
Customer O Mnti ton 



20 

H4S msrpn»l te<^n«:al comp4lsnc» c»fUr ak.l sr*ss. jfae/jyg /ifff /ntf f>gf/f> «>tffflfnfl rmtng (oar. u^by 

ft\ts^. »t\o*n p«tt»fn o{ sbs^nc* Itom worti •uog»rtv» o> U* erf d«p«ftd«Wrty. «(ti»n tsc*d 
pfoW«m» «««t(S oui c^»rs to solv* frtm d«monttrste$ concern for snbcpsbng r« 
^ n^^ds of cuatom^fs. twfcwsfd cn cocflfn'jnicslng idMS to e*i*r4 
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INTERVIEW RATING SHEET 



POSITION: APPLICANT: 

DATE: 



The Performance Skills to be 
evaluated 


Evidence 
sidll NOT 
present 


Evidence 
Some 
SKIU 
present 


Evidence 
Adequate 

Sidll 
Present 


Evidence 
Above 
Average skU 
present 


Evidence 
Superior 
skill 
present 


Insufficient 
evidence 
for or against 

skill 


(1) RISK-TAKING^NNOVATION 


O 


© 


© 


© 


© 




|(2> TEAMSWLLS 


O 


© 


© 


© 


© 




(3) LEADERSHIP 


o 


© 


© 


© 


© 




(4) PROBLEM-SOLVING SKJLL 


o 


© 


© 


© 







NOTES: 

Actu*i r*tingtorm us9dby 
mrtptoyr of «ngn««nf 



What Employers Want: 
A Summary 

❖ Learning to Learn 

❖ Listening and Oral Communication 

❖ Competence In Reading, Writing, and Computation 

❖ Adaptability: Creative Thinking and Problem Solving 

❖ Personal Management: Self-Esteem, Goal 
Setting/Motivation and Personal/Career Development 

❖ Group Effectiveness: Interpersonal Skills, Negotiation, 
and Teamwork 

❖ Organizational Effectiveness and Leadership 

Source: Workplace Basics: The Skills Employers Want, American Society for 
Training and Development arvd U.S. Department of Labor. 
Employment and Training Administration. 1988. 
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Classification of Taslis : 



1 . Can the task be SUBDIVIDED? 

Is n 



or 



Divisible, 

Unitary (IndrvtsiWe)'' 




What IS the GOAL of the task? 

/s n to optimize quality, 
or Maximize quantity? 

How are individual EFFORTS related to the team's performance? 

Is it Conjunctive • All team members must contnbute 
to the task 

Disjunctive • W one gets it, then a» get rt 
(eureka/non-eureka) 

Additive Rope tug, stuffing envelopes 

Compensatory - One person's extra effort makes up 
for another's reduced effort 

Discretionary - Team decides how individual efforts 
relate to team performance 




Wf^y Use Teams ? 



Teams are vital because all the following are 
divisible, optimizing, conjunctive tasks: 

❖ Effective meetings 

❖ Strategic planning 

❖ Implementing innovation 

❖ Designing continuous improvement projects 
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Most Teamwork Occurs through 
Face-to-Face Interaction in Meetings 



Meetings 



❖ Over 19 million meetings take place everyday in the U.S. 

❖ People spend over 1/2 their working life conducting, 
attending, preparing for and following up on meetings 

❖ Almost 1/2 of all meetings are considered unnecessary by 
the people who attend 

Good teamwork requires effective meetings! 




Impact of Team Development 



From: Katzenbach & Smith, The Wisdom of Teams: 
Creating the High-Performance Organization. 

Used with permission. 



Performance 
impact 
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A Team 
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Process Check 



❖ Monitoring of processes, especially processes 
which you own or are a part of, is necessary if 
improvement is desired 

❖ A process check Is a formalized way to do this 
monitoring and should be used at the end of all 
process related activities 

❖ There are many different ways to conduct a 
process check 

❖ A process check must focus on the process 

❖ Failure of a process does not infer any judgment 
about the quality of the team members 



^ I Process Check : Plus / Delta 

On one *postit\ place a + at the top and write a 
BRIEF comment below about ONE thing you found 
valuable and would keep for future sessions. 

On a second 'postit', place a A at the top and write a 
BRIEF suggestion below about how to improve ONE 
thing for future sessions. 

Post the notes in the place designated by the workshop 
facilitator when you are finished. 
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Ncrms 3x Communication 
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Learning a Process for 
Maintaining an Effective Team 



in this session, we will learn and practice a process for improving and maintaining the 
communication skills of the team. All of the issues and problems that can and do arise as a 
natural consequence of using teams need to be addressed by the team; preferably using a 
standard process The process presented in this session includes the development of social 
norms' which can be used to reduce, if not eliminate, the impact of these issues and problems on 
team pertom^ance. This is your opportunity to learn how ^ effectively (or affectively) ^^^^ 
many of your past complaints about working in groups: e.g.. 'He is always late or skips scheduled 
meetings ' 'ShTis never prepared.'. 'She never completes his part of me problem or assignment. 
•He always wants to copy my work and thaf s cheating', etc. Many of these behaviors were 
illustrated in the two videos used In this dass. Please do your best today to develop an 
understanding of this process. I want you to keep an open mind about the process and. 
hopefully, leam to use it to develop a culture in your team that Is harmonious and cohesive as 
well as productive ! Working in 'real' teams can be fun I 

NOTE- Social norms are the agreed upon behaviors, attitudes, values, etc. which hold 'societ/ in 
general and teams in particular together. These may be implicit or explicit but they MUST be 
commonly understood, reinforced, and taught (N.B. In the team environment, they aJso MUST be 
EXPLICIT reinforced and learned by ail team members. ). Sociologists believe that it is upon the 
•norms' that a society is built. They see these 'norms' as the 'glue' which holds society ( culture, 
subculture team, etc ) together . When establishing a team, the Code of Cooperation is one way 
to explicitly develop norms and serves to create a basis for organization and social interaction. 
When agreed upon norms start to fall apart or people disregard them and there is nothing to take 
their place there ceases to be cohesiveness. The disintegration of norms creates disorganization 
which may lead to a state of anomie. Then you are really in trouble! 



Communication S/r/V/s & Norms : ; 
Session Structure 

❖ Lecturette (15 minutes) 

❖ View a video; look for examples (25 minutes) 

❖ Break (until the next class period) 

❖ Promoters & Barriers to Effective Communication 
(25 minutes) 

❖ Developing iMorms (35 minutes) 

❖ Break (10 minutes) 

❖ Practicing Constructive Feedbacl^ (15 minutes) 

❖ Session Process Check (+/delta, 5 minutes) 

❖ Academic Joumal Entry (10 minutes) 
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Ten Commoindments'^ 



Help ecich other be rigkt not wrong. 

iookJor\'^cii^$ to mcike new loLecis work, 
not Jbr reasons they won't. 

3, rf in doiA^bL check It out! 

Don't make negative dSSiA^mptlons dbouX each other. 

Help each other win, and take pride in each other's victories. 

5. Speak i^ositivety aboat each other and 
akjoat i^O(A.r organization at every 0|ijportM,nity. 

6. Maintain a positive mental attitiA.de 
no tnatter what the circRfnstances 

7. Act with initiative anct courage, as ijit all depends on yon. 

8. Do everything with enth^siasna; it's contaqioiA^s. 

9. Whatever yon want; give it away. 

10. Don't losejaith. 

11. Have Jan! 



Consens 



us 



an agreed upon decision by all 
team members that reflects fuii 
exploration of a decision issue 
and does not compromise any 
strong convictions or needs ... 
often becomes new policy 
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OVERCOMING 

Group NO think 

❖ De-emphasize status and 

power differences between members. 

❖ Welcome outside viewpoints. 

❖ Encourage disagreement or clash of opinions. 



❖ Assign one member the task of 
being a devil's advocate. 

Ad^Ml Irom 8o«ng Cotmwicul Airpl«i« Group Trann{) hWanals 





82f 



Face-To-Face Communication 

100%— I 

55% 



0% 





37% 




■ 



8% 



Non-Verbals Tone of Voice Words (Verbal) 
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COMMUNICATION ROADBLOCKS 



• Directing • Moralizing 

• Interrupting fi ^ ®. * Persuading 



• Judging 

• Name Calling 




S^f 1 FIRST Seek the Intersection ' 



Their /, ^ \ Your 

/Position- r*^''^^^*'°" 'Position'; 



❖ First, see the problem from the other point of view ; 
really seek FIRST to understand ! 

❖ Second, identify the 'Intersection' 

( i.e., where the 'positions* clearly overlap ). 

❖ Finally, select ONE issue at a time from outside 
the Intersection' to discuss and resolve. 

• Select the issues that are 'closest' to the Intersection' 
and work 'outwards' from there : alternating between 
'their position* and 'your position'. 





Listening Skills 



❖ Stoptalking. 

❖ Engage in one conversation at a time. 

❖ Empathize with the person speaking. 

❖ Ask questions. 

❖ Dont interrupt. 

❖ Show interest. 

❖ Concentrate on what is being said. 

❖ Dont jump to conciusions. 

❖ Control your anger. 

❖ React to ideas, not to the speaker. 

❖ Listen for what is not said. Ask questions. 

❖ Share the responsibility for communication. 




Listening Techniques 



❖ Critical Listening 

• Separate fact from opinion. 

❖ Sympathetic Listening 

• Dont talk - listen. 

• Dont give advice - listen. 

• Dont judge - listen. 

❖ Creative Listening 

• Exercise an open mind. 

• Supplement your ideas with another person's 
ideas and vice versa. 




169 



Paraphrase for Understanding - 

Seek First to Understand, Then to be Understood * 

❖ Often we are either speaking or preparing to speak with 
the clear intent to reply, no? 

❖ Try listening with the intent to paraphrase what is being 
said by tlie other person. 

❖ To improve your understanding and to promote true 
consensus, paraphrase what was said by the speaker 

• until the speal<er agrees with your paraphrase ! 

❖ Then, thoughtfully prepare and deliver your response. 

❖ This is often referred to as 'empathic listening'. 

* Covey, Stephen R.. The 7 Habits ol Highly Effective People, (1989). Simon & Schuster, Habits 4 and 5. 



Talking Ciiips * 

❖ Each person selects one *totem' ( your pen is fine ). 

❖ If you want to talk, place your 'totem' in the center of 
the table ( or out in front of you ). 

❖ First come first talk, so to speak. 

❖ While your 'totem' is still in the center of the table, you 
can not talk again until 

• everyone's 'totem' is in the center of the table 

OR 

• the current discussion topic is completed. 

❖ When all of the 'totems' have been used or the current 
discussion topic is complete, the 'totems' are retrieved 
and the process begins anew. 

❖ Spencer Kagan descrbes a nunrt>er of communication regulators' including Talking Chips. 
Colored Chips. Response Mode Chips, and Paraphrase Passport in Chapter 13 of his book. 
Cooperative Learning (1992. San Juan Capistrano. CAi Resources lor Teachers) 
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Constructive Feedback 



IS 



❖ Communication to a person (or group) regarding 
the effect that a person's behavior has on 
another person or on the group 



❖ Perceptions, feelings, and 
reactions to the message 




Constructive Feedbaclc 



❖ You are an expert on 



other people's behavior 



your feelings 



❖ You are NOT an expert on 



your behavior 

other people's feelings 
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5^ ^ How to Give Constructive 

Feedbacli 



1. "When you 



2. "I feel . . ." 



Start with a 'When you , . " 
statement that describes the 
behavior without judgment 
exaggeration, labeling, attribution, or 
motives. Just state th© facts as 
specifically as possible. 

Tel! how their behavior affects you. 
If you need more than a word or two 
to describe the feeling, ifs probably 
just some variation of joy, sorrow, 
anger, or fear. 



3. ''Because I 



Now say why you are affected that 
way. Describe the connection 
between the facts you observed and 
the feelings they provoke in you. 



From Scholles. Peter R.. The Team Handbook, Joiner Associates (1988) 



^ J How to Give Constructive 
Feedback (cont,) 

4, (Pause for Discussion) Let the other person respond. 



5, "I would like , , Describe the change you want the 

other person to consider ... 



6, "Because , , " ^'^^ ^^^^ ^® change will 

alleviate the problem. 



7, "What do VOU think " Usten to the other person's response. 

^ ' * " Be prepared to discuss options and 

reach consensus on a solution. 



I 
I 

B 
i 
I 
I 
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How to Give Constructive 
Feedbacic Example 



1. "When you..." 

2. "I feel . . 

3. "Because I . . ." 

4. (Pause for Discussion) 

5. "I would like . . 

6. "Because . . ." 

7. "What do you think . . 



*When you are late for team meetings. 
I get angry ... 



... because I think it is wasting the time of 
all the other team members and we are 
never able to get through all of the 
agenda items.** 



"I would like you to consider finding some 
way of planning your schedule that lets 
you get to these team meetings on time. " 

"Because that way we can be more 
productive at the team meetings and we 
can all keep to our tight schedules." 




To Realize the Benefits of a Team Culture 
Requires a Change in Management Behavior 



FROM 



Directing 



Competing 



Relying on Rules 



Using Organizational — 
Hierarchy 

Consistency/Sameness 

Secrecy 

Passive 



Isolated Decisions 
People Costs 



Results Thinking * 



IQ 

Guiding 
Collaborating 
Focus on the Process 
Using a Network 

Diversity/Flexibility 
Openness/Sharing 
Risk Taking 
Involvement of Others 
People Assets 
Process Thinking 




Promoters & Barriers of 
Effective Communication 



❖ Goals 

• Anticipating Potential Problems and 
Establishing Corrective Norms 

• Make team members aware of what things 
bother other team members 

• Organize potential problems into major groups 

• Develop a strategy for addressing potential 
teaming problems 

• Develop a knowledge of Team Processing Tools 
( see the Appendix for detailed information ) 
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(^^ ^ Force Field Analysis 



Purpose 

A torce field analysis helps teanns find out what is dnving, slowing, or not allowing change. The tool 
helps a team to work together, to lind a starting point Irom whtch to take action, and to show both sides 
ot the change issue. 

Steps 

1. On a board or large piece ol paper draw a vertical line down the middle and a horizontal line across the 
paper near the top 



2. Label the left column Promoling and the T\g\,\ column Preventing 

3. Brainstorm entries lor the lett hand column. 

4. Brainstorm entries tor the right hand column. 



ProtMling 
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5^ J Brainstorming and Affinity Processes 



Post-its 



❖ Purpose: 

^ To organize a large set of items into a smaller 

set of related items. 
^ Guidelines: 

^ The rules of brainstonning are followed but 
each idea is written (in 7 words or less, 
tnclading a noun and a verb) on a self- 
adhesive Post-it note or card 

^ Team members siUnth move the Post-it cards 
around to form closely-related idea groups 

^ If disagreement exists when grouping, make 
copies of the contested card and place in more 
than one group 

Label each group with a header card which 
clearly identifies and reflects the theme of the 
cards 

^ If there are single idea cards that don't fit well 
with the other ideas, have the team dedde if 
they should be kept (they may be excellent 
ideas thought of only by one person) 




Brainstonning 



Affinity 



Header Cards 













ill 
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Modified Nominal Group 
Technique 

Purpose 

Modified nominal group technique is a technique to help a teann or group quickly reduce a large list of 
items to a smaller number ot high priority items. The process elicits a high degree ot team agreement 
And promotes team ownership. Thss tool is similar to nominal group technique but not quite as involved. 

Step« 

Step 1 

Count the number ot items on the list and divide by three. This is the number ot votes each person has. 
(Round fractions oJt to the lower number.) If the ttenrtt number more than 60, do rwt go over a vote total 
ot 20. Vote totals ot more than 20 are hard to nrwnage. Give each team member as many colored dots 
as she/he has votes. 
Step 2 

Have each person use his/her votes (colored dots) to select the r.ems he/she wants to keep. Whtle 
each person can vote for any item, it is a good idea to limit the number ot votes any one item can 
receive from a singte person to three. Note: the team can decxJe it they want to alkjw more or less 
mutt 'pte voting. 
Step 3 

List alternatives m their new prioritized order 
Step 4 

Critically discuss the top afternativos in order to reach consensus. Eliminate those that are outside the 
control otthe team 
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ASU COPY CENTERS 



CLASS NOTES 



] 



SEMESTER 

INSTRUCTOR 

COURSE 

PRICE 

TAX 



$. 



TOTAL 



S 



Education Copy Center 
■ (EDB 25 Payne Hall) 
965.7252 
M - F, 8 - 5 pm 



Administration 
Copv Center 

(ASB 203C) 
.965.4422 
M - F, 8 - 5 pm 



Engineering Copy Center 
(ECF 103) 
965.6071 
M - R 8 - 5 pm 



Memorial Union 
"ASU Fast Copy" 
(MU l6bv.AJvlEX) 
965.7251 
M - R 8 - 5 pm 



Business Adminstration 
Copy Center 
(BAC North Lobby) 
965.0730 
M - F, 8 - 5 pm 

Computing Commons 
Copv Center 
(CPCOM 215) 

965.8932 
M - R 8 - 5 pm 
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